A Texas colleague recently wrote the following:

We have several educational Software Packages such as A+ Learning, Renaissance Place, and Study Island to name a few. Some are Web-based and some are not. How do you evaluate the effectiveness of these applications? I think we spend a lot of money with no effort to see if it is effective. I am interested in any thoughts that anyone has on this matter.

Here is a response from Dr. Richard Smith, co-blogger of the Education and Technology Today Podcast at the University of Houston at Clear Lake:

Here is a basic reliable method to determine your measurable goals for instruction in the classes in which the software is being used.

1) Determine which instrument will be used to see if the goals have been met (a standardized test for that subject, or perhaps a test which your teachers create that determines whether the students have gained the knowledge taught, etc).

2) Set up three test groups (classes)preferably all taught by the same teacher (ideally with students randomly assigned to each class, but we know that won’t happen because of scheduling problems, etc.).

Class 1 – the teacher will teach using the software to be evaluated. Give a pretest and posttest given to the students.

Class 2 – the teacher will teach BUT NOT USE the software to be evaluated. Give a pretest and posttest.

Class 3 – the teacher will use the software to be evaluated WITHOUT a pretest but WITH a posttest.

Make certain that you do not simply do a pre and post test on one group of students.  That is a non-reliable method of determining the effectiveness of your software (although widely used).

If you have an evaluation specialist in your district who can work with statistics, work with that person to do the posttest analyses.  If not, find a specialist at a local university or community college.

How do you evaluate software in your district?

Subscribe to Around the Corner-MGuhlin.org

Everything posted on Miguel Guhlin’s blogs/wikis are his personal opinion and do not necessarily represent the views of his employer(s) or its clients. Read Full Disclosure