Jun 052013
 

Recently had a problem where awstats stopped processing log entries from one of my sites while the others worked fine. While some of these items don’t make sense to check in this instance, you may want to give them a look if you are having problems.

Check a few things first:

  1. Do you have enough disk space? – Yes
  2. Is the log file getting updated? (I was shipping logs from one server to another for processing) – Yes
  3. Has the log format changed? (Take a look at historical logs if available to make sure. Also check your /etc/awstats/awstats.config.conf file to make sure it is the same here) – All good
  4. Files still getting written to your data dir? Do you have any contents in any missing files? (/var/lib/awstats/awstats{month}{year}{day}.txt) – New files are there for working sites, nothing present for the failing site.
  5. Turn on showing dropped records and what did it say? – Dropped record (method/protocol ‘rtmp’ not qualified when LogType=S):
  6. Have you made any changes to the awstats application recently? – Yep….wait what?

It seems at some point we upgraded awstats to the 7.0 branch, and had previously made customizations to the /usr/local/awstats/wwwroot/cgi-bin/awstats.pl to handle our FMS logs. We did it according to the instructions here: http://www.wowza.com/forums/showthread.php?163-Log-Analysis

When we did the upgrade, we overwrote our changes, an this one log stopped processing.

So, I made the following change to /usr/local/awstats/wwwroot/cgi-bin/awstats.pl around line 18158 to make this block:

                        ( $LogType eq 'W' || $LogType eq 'S' )
                        && (   uc($field[$pos_method]) eq 'GET'
                                || uc($field[$pos_method]) eq 'MMS'
                                || uc($field[$pos_method]) eq 'RTSP'
                                || uc($field[$pos_method]) eq 'HTTP'
                                || uc($field[$pos_method]) eq 'RTP' )

look like this:

                        ( $LogType eq 'W' || $LogType eq 'S' )
                        && (   uc($field[$pos_method]) eq 'GET'
                                || uc($field[$pos_method]) eq 'MMS'
                                || uc($field[$pos_method]) eq 'RTSP'
                                || uc($field[$pos_method]) eq 'RTMP'
                                || uc($field[$pos_method]) eq 'RTMPT'
                                || uc($field[$pos_method]) eq 'HTTP'
                                || uc($field[$pos_method]) eq 'RTP' )

I then started reprocessing old logs like this:

/usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=fms -LogFile=/oldlogs/access.30.log -showdropped -showcorrupted

And all is good in the world again.

Dec 232010
 

Hello everyone,
I’ve been saying that I was going to post this for a long time, ever since my presentation at Fusion 2010: http://www.desire2learn.com/fusion/ and now I finally have.

I have written up two pages on code I developed for use with PHP and D2L’s web services. I have been using it at my work for the last year or so, and I finally polished it up a little, and created some installation and usage instructions. I’m not finished yet, but it is a start.

If you use D2L and want to extend it through Web Services (you need to purchase the web services package from D2L, and have it installed first), I encourage you to read over these pages:

If you have any comments, please post them below. Feel free to email me with any questions or problems you run into while working with the API or the testing framework. I’m eager to improve the code for everyone.

May 072010
 

Just a quick announcement that I’m working on a new programming project called The Director!. It is essentially a url redirection/shortening service written in PHP and utilizing the apache rewrite module to do the really cool stuff.

Those in the know realize that creating such a service is just a few lines of code in the vhost entry for apache, and then a few lines of code in a php script to do the redirection, but the goal here is to make it easy to use management interface with a bit of simple stat tracking.

Hoping to have version 0.1 out this weekend (depending on how my fence construction goes this weekend).

You can view more information here: http://wiki.cornempire.net/thedirector/start

Dec 062009
 
Edit: I’ve made a new post regarding Hesk and these enhancements for the new version 2.2. You can read it here: http://cornempire.net/2010/07/31/hesk-enhancements-for-version-2-2/

I came across the free, open source Hesk Helpdesk (http://www.hesk.com) while I was looking for a light weight help system for a project that was starting up. The project has since been put on hold, but Hesk is still a great application for a helpdesk/knowledge base. Hesk is offered as a free downloadable PHP script, as well as a Hosted Version if you do not have a server. The hosted version will cost you $20 US a month. You can also by a license for the downloadable version for $39.95 US to remove the Hesk branding.

While this application is great…

  1. Allows unlimited staff
  2. Granular permissions to the system and help desk categories
  3. Excellent knowledge base built in
  4. Innovative system that doesn’t require user accounts for clients

It is lacking a few features. I plan to use this application to respond to clients from a number of different projects. As such, it would be nice if they didn’t see ALL of the problem types when submitting tickets. Because Hesk is open source, we can make a few tweaks to get directly to a specific problem type.

Hesk also allows custom fields for your tickets, but it shows all custom fields on all problem types. This can get confusing for users in a multi-project environment. I’m developing a little hack to get around this limitation.

I’ve outlined both of these changes required on my wiki, you can take a look here.

Oct 222009
 

Just surfing the web and came across this excellent site with advanced awstats configuration information. As you all know, I do enjoy using awstats, and have had to modify it in the past.

Well, this site has lots of cool scripts and modifications to make awstats more useful: http://www.internetofficer.com/awstats/

My favourite modifications so far are:

And don’t forget to check out my own tutorial on adding robots (Blog Post/Wiki Entry)

Jul 102009
 

I just noticed this today. I monitor a number of sites and servers using the great Xymon system monitoring tool (Linky). It is based on early Big Brother code. Anyway, today I noticed that the graphs were all blue. Something like this:

Bad Xymon Graph

Bad Xymon Graph

I’m using Xymon 4.2.3 on the servers, and both servers show the graph like this using Firefox 3.5 on Linux. Google Chrome development version, SeaMonkey 1.1.5, Epiphany and Galeon all display the graph correctly.

I have no solution, but I thought I’d post my experience.

Jul 072009
 

Note:

Instead of editing my previous post to pieces, I thought it might be best if I repost my script with a better description of how it works. I’ve written up documentation of the script on the CornEmpire Software wiki which is available at http://wiki.cornempire.net. The documentation directly related to this script is available at http://wiki.cornempire.net/doku.php?id=d2l:bulkdeactivate

Introduction

One of the often requested features is to be able to bulk deactivate courses in D2L after they have completed. You can currently deactivate courses, but this is a one at a time effort. This has become more important as version 8.3 of D2L brought along a new My Courses widget, which allows users to see updates from each of their courses. Unfortunately, when course access ends, they can still see the updates. This has caused confusion for many users.
Another reason to deactivate courses is minimize the amount of clutter in the view of users. If course access has ended, and they cannot access the course, there is no need for the course to remain active (this could vary depending on how your roles are configured). A way around this is to script the bulk deactivation of courses. Through the use of some Javascript coding, we can instruct the web browser to visit each course, and deactivate it for us.

See the script in action.

Would you like to see more? I have the script, and it’s documentation available on this page: http://wiki.cornempire.net/doku.php?id=d2l:bulkdeactivate

If you have any questions, please post them below.

Jun 092009
 

I’ve been recently doing some Javascript coding and ran across an error in IE that I wasn’t noticing in Firefox (well…there are hundreds of these, but this one is a show stopper).

It is a small validation script for a form to do some checking before submitting.  It’s pretty simple, take a look:


function validateForm(){
	/*
	 * First, get the fields.
	 */
	var name = document.getElementById('name').value;
	var title = document.getElementById('title').value;
	var phone = document.getElementById('phone').value;
	var email = document.getElementById('email').value;
	var description = document.getElementById('description').value;
	var cost = document.getElementById('cost').value;
	//pic1 = document.getElementById('pic1').value;
	//pic2 = document.getElementById('pic2').value;
	if(name == ""){
		alert('You must enter your name.');
	}
	else if(title == ""){
		alert('You must enter give your ad a title.');
	}
	else if(phone == "" && email == ""){
		alert('You must enter a phone number or email address.');
	}
	else if(description == ""){
		alert('You must enter a description of the item you want to buy, sell or trade.');
	}
	else if(isNaN(cost)){
		alert('You must enter a numeric cost, without the $ symbol.');
	}
	else{
		document.getElementById('submit').style.display = 'block';
		alert('Please press the submit button below.');
	}
}

Executing the script on a blank form in FF would give the correct response, Enter in a name. In IE, I would get an error on Line 15 about Object Expected (in the script, line 15 is actually a commented out line, so that error isn’t very helpful).

I then remembered something about IE automatically creating variables from a form by the name/id of the form field on the page. Since my variable name, and field name were the same, it would try to compare its self created object against my test, instead of the variable that I was assigning, thus creating an error.

To work around this, I had to rename all of my variables (I could have also renamed all of my fields):


function validateForm(){
	/*
	 * First, get the fields.
	 */
	var bsname = document.getElementById('name').value;
	var bstitle = document.getElementById('title').value;
	var bsphone = document.getElementById('phone').value;
	var bsemail = document.getElementById('email').value;
	var bsdescription = document.getElementById('description').value;
	var bscost = document.getElementById('cost').value;
	//pic1 = document.getElementById('pic1').value;
	//pic2 = document.getElementById('pic2').value;
	if(bsname == ""){
		alert('You must enter your name.');
	}
	else if(bstitle == ""){
		alert('You must enter give your ad a title.');
	}
	else if(bsphone == "" && bsemail == ""){
		alert('You must enter a phone number or email address.');
	}
	else if(bsdescription == ""){
		alert('You must enter a description of the item you want to buy, sell or trade.');
	}
	else if(isNaN(bscost)){
		alert('You must enter a numeric cost, without the $ symbol.');
	}
	else{
		document.getElementById('submit').style.display = 'block';
		alert('Please press the submit button below.');
	}
}

After this, it runs fine in both browsers. I remember hitting my head off the wall for a LONG time when I originally encountered this error, so hopefully this will save someone else some time.

Feb 092009
 

Introduction

The picture library built into Desire2Learn doesn’t allow for instructors to create their own libraries. Rather, all images are shared across all courses. I wrote this bit of javascript to simulate and extend the idea of a picture library in D2L (this code can also be used for any HTML page). There are two parts to the library:

The Widget Code (or HTML code for any page)
This code is the skeleton of the library. The javascript code modifies this code in order to display the images.
The Javascript Code
This is the brain of the library. It must be installed somewhere locally and linked to the widget/HTML page. There is a configuration section at the top of the file where you can configure the library.

Here is the widget/HTML code that must be displayed on the pages.
You should not edit this code as it will cause the library to malfunction.


<html>
<head>
	<script src="picturelibrary.js" type="text/javascript"></script>
</head>
<body>

	<div id="imgwdgt">
		<img src="piclibimages/test.png" id="imghldr" />
		<p id="imgcptn"></p>
	</div>
	<script type="text/javascript">
		startLibrary();
	</script>

</body>
</html>

Here is the javascript code: picturelibrary.js
You will need to right click this link and save the file to your computer before you can upload it to your D2L shell/website.

Notes About The Code

Not all the variables need to be set a certain way in order for the code to function. However, it is a good idea to set up any variables that you don’t want to use to be false. Some variables require others, so read the comments in the code carefully. I’ve also included them below:


/**
	This script presents a picture library to the user.  It has several configurable
	options which are outlined below.
	With this script, you can:
		1) Present a slide show to users
		2) Present a navigatable interface for users to scroll through the library
		3) Release an image to them based on a set date/time
		4) Release a random image from the library
**/
/**
	=== CONFIG START ===
	Here you need to specify the path to this script.  To do this you must go to the
	Edit Course section of the site, and go to Manage Files.  Then, copy the URL for
	this javascript file.  Paste the URL into the variable below.  Then remove the
	filename at the end of the URL "picturelibrary.js".  Replace the file name with
	"piclibimages/" (no quotes)
*/
var path = "http://online.mun.ca/content/Sandbox/TT/Sp07/piclibimages/";

/**
	List your images here.  By default, it will look under the piclibimages folder
	in your course files.  Change the path above if you want to load from a different
	location.

	Place the file names in the array below.  A comma must be placed after each entry
	except for the last entry in the list.
*/

var images = new Array(
	"dataspace-code.gif",
	"MedSchoolCrestSmall.gif",
	"crest-toothpaste.png"
);

/**
	List your captions here.  

	If you enable captions below, you must have at least as many captions as you have images
	above or you will probably get a javascript error when the script scrolls through the
	images.  You can have blank captions, denoted with "" in the array below.
*/
var captions = new Array(
	"The icon I created for the phpLive -> Dataspace integration.",
	"Medicine School Crest",
	"Crest Toothpaste"
);

/**
	This array contains dates for the date release function of this script.  If you do not
	want to use date release, you can leave this blank.

	Each entry goes with an entry in the images array and should be formatted with
	the startdate-enddate for the image.  Only one image can be displayed at a time
	with this method.
	A sample entry is: 200809190000-200809222359
		This will allow the	image to be viewed on Friday the 19th until Monday the 22nd.

	If you want your images	to rotate through a day, you can place a * for the date and
	just include the time.
	A sample entry is: *0000-*1200
		This will allow your image to appear on the site between midnight and noon every
		day.
*/

var releasedates = new Array(
	"*0000-*1230",
	"*1231-*1300",
	"*1301-*2359"
);
//var releasedates = new Array(
//	"200809170000-200809182359",
//	"200809190000-200809202359",
//	"200809210000-200809222359"
//);

/**
	Here you can set your options for the script.
	resize = controls if images are automatically resized.  If true, width and
			 height must be set.  These are measured in pixels.  Be careful using this
			 as it will likely distort your pictures horribly.  It is much better to
			 resize your pictures before uploading them to the system.
	navigation = controls if users can navigate through images, or if they see
				 a loop of images.
	imagepause = the amount of time in milliseconds that an image appears before
				 changing.  Only used if navigation is false. 4000 = 4 seconds
	captionsdisplay = if this is set to true, the caption is pulled from the above array
			   		  and displayed with the picture.
	daterelease = Shows an image on a specific date/for a specific time.  The datearray
			      must store the dates in the format yearmonthday-yearmonthday or *time-*time.
			      the first date/time being when the image is displayed, and the second
			      when it ends.
	imagestyle = This is a CSS style tag that will be added to the image tag.
	captionstyle = This is a CSS style take that will be added to the caption.
	widgetsytle = This is a CSS style tag that well be added to the div around the image
				  and caption.
	randomstart = This variable controls whether or not the script begins on a random image.
	singlerandom = If true, this displays a single random image when the page is loaded.
*/			      

var resize = false;
var width = "";
var height = "";
var navigation = true;
var imagepause = "5000";
var captionsdisplay = true;
var daterelease = true;
var imagestyle = "margin: 0 auto; display: block; padding: 2px;";
var captionstyle = "margin: 1px; text-size: 10px; text-align: center; border: 1px dashed gray;";
var widgetstyle = "border: 2px solid black;";
var randomstart = true;
var singlerandom = false;

/**
	=== CONFIG END ===
	This is the end of configurable items for this script.  Do not edit the code below.
*/

Also of note is the use of CSS styles. I cannot figure out how to apply an arbitrary CSS style string to an HTML element. I can change specific things, but I cannot allow any style tags. If anyone knows how, please contact me.

Because of this limitation, the CSS code will only recognize:

  • border
  • text-align
  • font-size
  • margin
  • display
  • padding

You can add in your own code to the bottom by editing the doStyle function, or you can apply your own styles to the raw widget/HTML code. You must leave the style variables blank in this script.

Final Thoughts

I hope you find this useful, either for your D2L course, or for your personal homepage. I’ll probably post updates from time to time on this script depending on the feedback I receive. If you have any comments, you can leave them below. Take a look at some of the variable configurations below to see examples of how the code can be used.

Examples

This configuration allows you to display a slide show that updates every 5 seconds. It has a random starting point, and has captions enabled.
var resize = false;
var width = “”;
var height = “”;
var navigation = false;

var imagepause = “5000”;
var captionsdisplay = true;
var daterelease = false;
var imagestyle = “”;
var captionstyle = “”;
var widgetstyle = “”;

var randomstart = true;

View Example 1


This configuration allows you to navigate the images and it also displays captions. Styles are also applied to the image, captions and box around them both.

var resize = false;
var width = “”;
var height = “”;
var navigation = true;
var imagepause = “5000”;
var captionsdisplay = true;

var daterelease = false;
var imagestyle = “margin: 0 auto; display: block; padding: 2px;”;
var captionstyle = “margin: 1px; font-size: 10px; text-align: center; border: 1px dashed gray;”;
var widgetstyle = “border: 2px solid black;”;
var randomstart = false;

View Example 2


This configuration shows a different image depending on the time of day. To best simulate Newfoundland weather. No captions are displayed. Not all images are displayed at any part of the day.

var resize = false;
var width = “”;
var height = “”;
var navigation = false;
var imagepause = “5000”;
var captionsdisplay = false;

var daterelease = true;
var imagestyle = “”;
var captionstyle = “”;
var widgetstyle = “”;
var randomstart = false;

var releasedates = new Array(

“*-*”,
“*0800-*0830”,
“*0926-*1000”,
“*1001-*1030”,
“*0831-*0855”,
“*0546-*0759”,

“*1031-*1130”,
“*1146-*1200”,
“*-*”,
“*1201-*1230”,
“*-*”,
“*-*”,

“*-*”,
“*1131-*1145”,
“*-*”,
“*1231-*1300”,
“*-*”,
“*-*”,

“*1301-*1500”,
“*-*”,
“*-*”,
“*-*”,
“*-*”,
“*1501-*1600”,

“*1601-*1700”,
“*1701-*1800”,
“*1801-*1900”,
“*1901-*2000”,
“*2001-*2229”,
“*2230-*2330”,

“*0000-*0030”,
“*0431-*0500”,
“*0031-*0130”,
“*2330-*2359”,
“*0501-*0545”,
“*0131-*0300”,

“*0301-*0330”,
“*0331-*0430”,
“*0000-*0000”,
“*-*”,
“*-*”,
“*-*”,

“*0856-*0925”,
“*-*”
);

View Example 3


This configuration shows a single image. It is random from the list of images.
var resize = false;
var width = “”;
var height = “”;
var navigation = false;

var imagepause = “5000”;
var captionsdisplay = true;
var daterelease = false;
var imagestyle = “”;
var captionstyle = “”;
var widgetstyle = “”;

var randomstart = true;
var singlerandom = true;

View Example 4


This configuration shows a slideshow with a 5 second pause, and a random start.
var resize = false;
var width = “”;
var height = “”;
var navigation = false;
var imagepause = “5000”;
var captionsdisplay = false;

var daterelease = false;
var imagestyle = “”;
var captionstyle = “”;
var widgetstyle = “”;
var randomstart = false;
var singlerandom = false;

View Example 5