Sunday, December 9, 2007

Discuss how the following topics impacted your project, how they are of value to ID, and how you see them being used in the career you wish to enter

Introduction

The purpose of my Instructional Design project was to design a reference chat training program for Reference Department Interns at Jackson Library, UNCG. This process was documented in a blog dedicated to this project (http://christiesidblog.blogspot.com/). There are a number of activities that we have explored over the course of the semester that have helped me develop this reference chat program. They include performing a needs assessment and task analysis, considering the ABCD model of goals and objectives, constructing an alignment diagram, developing instructional strategies, and finally, deciding how assessment and evaluation will be handled. Each of these will be discussed in more detail here.

Learning Theories

The type of learning theory that is most applicable to this instructional design is cognitivism. This is learner-centered instruction. The interns actively participate in the learning process and as a consequence, they learn as they actually perform the task. I believe that it is very important to keep the different learning theories in mind during any instructional design process. The type of theory will determine the method of instruction, so this is critical to the ID process. It seems to me, however, that cognitivism lends itself more to the types of classes that reference librarians typically teach.

ADDIE

ADDIE is an acronym that stands for Analysis, Design, Development, Implementation, and Evaluation. Employing the ADDIE model in the development of a program or course can assist developers in instituting a learner-centered approach rather than a teacher-centered approach, making the program more applicable and meaningful for learners (Peterson, 2003). A variation of ADDIE was used for this Instructional Design project.

This particular ID project, IM-based reference chat instruction, used needs assessment (Analysis), learner and contextual analysis (Analysis), task analysis (Analysis), identification of goals and objectives (Design), an alignment diagram (Design), instructional strategies (Design)(Development)(Implementation), and assessment and evaluation (Development), and training evaluation (Evaluation). All aspects of ADDIE were covered in this instructional design.

I do not think that a reference librarian would go through this process for every single class that they teach, the classes are too frequent and too varied. I can see this process being used, however, to design instruction for representative samples of classes and the results being transferred to other classes. Also, libraries are becoming more involved in developing instructional technologies which are being used by instructors for both on and off-campus courses. Using ADDIE would be a great way to develop any courses in which new technologies are being considered to ensure that the proposed technology is meeting a genuine need of the course.

Needs Assessment

According to Roger Kaufman, a needs assessment provides a process for defining the gaps between current and desired results, and providing the justification for identifying and choosing the ways to close those gaps. Because the Reference Department already has a training system in place for teaching Reference Interns how to use the reference chat services employed by Jackson Library, this needs assessment afforded me the opportunity to determine where the gaps in the existing training are. Librarians teach courses that are varied in nature. It will be necessary to them to assess the needs of any course for which instruction is being designed.

Learner and Contextual Analysis

Questions designed to address learner and contextual analysis were built into the questionnaire that was distributed as a part of the needs assessment. Learner and contextual analysis questions help a person designing instruction determine the most effective modes of delivery, methods of instruction and effective use of media attributes for a given group of learners (Smith, 2001). As was pointed out in our readings, this is especially important when considering distance education instruction. This is an issue for both professors and for librarians who provide specific instruction for these classes.

Task Analysis

Hannum defines task analysis as the process of identifying the various tasks a person must perform while completing a job, determining which of these tasks require training and identifying the component knowledge and skill that a person must possess in order to perform each task in an acceptable manner.

A behavioral task analysis was performed for this project. A list of all activities that reference Interns are expected to perform as a part of their regular job duties was created, a task selected for this ID project (IM-based reference chat instruction), and that task was further divided into subtasks. If the task analysis is omitted from the design process, there is no way to ensure that the most logical area has been chosen upon which instructional design efforts should be focused. Given that each class will vary greatly in its scope and content, a task analysis is essential in every instance for the instructional design process.

Goals and Objectives

Distinguishing between the goals and objectives proved more problematic that I would have imagined. The most helpful resource we were given to help us with our goals and objectives was the PowerPoint presentation by Dr. Bob King. Even though this presentation demonstrates an alignment diagram and is therefore placed with the alignment diagram readings, I believe it would have been very beneficial to watch this presentation with the previous week’s readings on goals and objectives. I really did not grasp the distinction until I watched this presentation.

In the presentation mentioned above, Dr. King recommends writing a goal, which is an abstract, broad statement of intention, for each identified subtask. Then more concrete and measurable objectives are written for each goal. The ABCD framework is used to clarify each objective. This was the process used to identify goals and objectives for this assignment. The ABCD model of goals and objectives involves determining who the Audience is that the training is aimed at, what Behavior is desired of the interns performing this task, what Conditions the task will be conducted in, and the Degree to which it must be done.

Without goals and objectives, there is nothing to target with your instruction. You are teaching aimlessly. Librarians have a very limited amount of time in which to deliver effective instruction. Defining goals and objectives is necessary in order to make the most effective use of their instructional time.

Alignment Diagram

The alignment diagram was fairly straight forward after watching the aforementioned presentation by Dr. King. The biggest challenge was getting the diagram posted to my blog. It was an enlightening experience though. I now feel confident that I can convert and post a Word document onto a blog. An alignment diagram ensures that the instruction being designed meets the identified need. The diagram also provides the librarian, or anyone designing or providing instruction, a visualization of the entire process.

Instructional Strategies

A procedure strategy was determined to be the most appropriate for this particular training since reference chat is an activity that the Interns will actually have to perform on a daily basis. This involved identifying the knowledge structure, a presentation, an opportunity for exploration, practice with feedback, and learner guidance. Evaluation of the instructional process by the Interns has been built into the last step of this instructional design component. The instructional strategies section of this project involves three of the 5 components of ADDIE making it critical to the instructional design process. While this encompasses only part of the design process, it contains all aspects of development and implementation.

Assessment and Evaluation

Because this training involves a procedure that the Interns will actually have to perform during the regular course of their duties, the assessments are necessarily authentic in nature. It only makes sense that the intern be assessed while actually performing the task in question. Evaluation in this case involves mastery learning. The intern will receive instruction until they can demonstrate that they are proficient in the task. Anything less would be unacceptable since this is a required part of their job.

It was not until the very end of this project that I came to realize that this evaluation is different than the evaluation included as a part of ADDIE. Evaluation in ADDIE refers to the evaluation of the instruction, rather than the evaluation of the learners and whether or not they met the objectives specified. Still, discovering whether or not the learners have met the specified objectives helps the instructor to determine if the instruction was effective, so I suppose that they are related in a way. I believe that both types of evaluation are critical to the design of effective instruction.

Assessment and evaluation of the learners themselves is difficult, if not impossible, when instruction is being provided in one session by reference librarians. Even evaluation of the instruction would be difficult. In this particular case it worked because the learners are Interns who will be working in the Reference Department for at least one semester. I’m not sure how these areas would be addressed for a typical bibliographic instruction session. That would take a lot of planning on the part of the instructional designer.

Describe how you plan on putting your efforts into action

I chose to design a training program for reference chat because it is a tool currently used by the Reference Department at Jackson Library. Jackson Library Interns currently receive training in reference chat during the initial 10 hour orientation training session for new Interns at the beginning of each semester. The current training, however, takes place during one hour-long training session. While each of the goals are touched upon in the existing training, they are not discussed as thoroughly as they are in the training program that has been developed here. I plan on sharing this instructional design plan with Steve Cramer, the reference Intern coordinator. He is in charge of all Intern training. It will really be up to him where to take it from there.

How do you forsee using the tools, processes and technologies we've explored going forward?

Truthfully, I took this class because I thought it would help me secure a job, but I was not expecting to actually enjoy it. It has made me rethink the direction in which I am heading with this Library and Information Science degree. I came into this program, as I believe many people do, focusing on the Library Science component of the degree. I am becoming more and more interested, however, in the Information Science side of the degree.

I knew from my past teaching experience that I loved to teach, but I didn’t really know how this interest in instruction could be used outside of the public school classroom. The emphasis on technology and distance education in this class has opened my eyes to a world of possibilities.

Training Evaluation

Once the Interns have completed the IM-based reference chat training program and all of their comments on the effectiveness of the training have been examined, an effort will be made to re-evaluate the training program. Any changes that are deemed necessary will be incorporated into the instructional design for the next group of Interns.

Monday, December 3, 2007

Assessment and Evaluation

Goal 1 - The Intern should be able to log into both reference chat services.
Objective - Interns will demonstrate that they can log into both Velaro and Trillian
Assessment - At the end of the training period each Intern will be asked to log into both chat services when they come into work.
Evaluation - If the Intern demonstrates that they can successfully log into both chat services, he/she will be checked off on that particular objective. If not, they will receive an additional training session to help them with the skill, and they will be assessed again the following week. This will continue until the skill is mastered.

This is an authentic assessment that will be given informally to individual Interns. The assessment will be given at the reference desk computer where the interns work.

Goal 2 - The Intern should be able to use the tools available through the reference chat services.
Objective - Interns will demonstrate proficiency of the Page View feature.
Objective - Interns will demonstrate proficiency of the Page Push feature.
Objective - Interns will demonstrate proficiency of the Transfer Files to Customers feature.
Assessment - At the end of the training period, Interns will be given an assessment to determine their proficiency using the chat service tools. The intern will be asked to use each of these tools while working at the reference desk. The instructor will send the Intern inquiries using the chat services and the Intern will have to use the aforementioned tools in order to satisfactorily respond to the inquiry.
Evaluation - If the Intern demonstrates that they can use the required reference chat tools, he/she will be checked off on that particular objective. If not, they will receive an additional training session to help them with the skill, and they will be assessed again the following week. This will continue until the skill is mastered.

This is an authentic assessment that will be given informally to individual Interns. The assessment will be given at the reference desk computer where the interns work.

Goal 3 - The Intern should be well-versed in reference chat etiquette
Objective - Interns will demonstrate that they can work on a reference chat problem and keep the patron informed of their progress.
Objective - Interns will demonstrate that they can suggest solutions without abruptly telling the patron what to do.
Objective - Interns will demonstrate that they can react calmly when confronted with an irate patron.
Assessment - As with the reference tool assessment, this will be given to the Intern when they are working on the reference desk. The instructor will send the Intern inquiries using the chat services and the intern will have to respond accordingly.
Evaluation - If the Interns demonstrate that they are well-versed in reference chat etiquette, he/she will be checked off on that particular objective. If not, they will receive an additional training session to help them with the skill, and they will be assessed again the following week. This will continue until the skill is mastered.

This is an authentic assessment that will be given informally to individual Interns. The assessment will be given at the reference desk computer where the interns work.

Goal 4 - The Intern should be able to prioritize reference chat in relation to other Intern responsibilities.
Objective - Interns will demonstrate that they can properly prioritize chat questions when getting questions from multiple sources.
Assessment - The Intern will be assessed on this competency while working at the reference desk. The instructor will schedule a time when he/she can work at the desk with the Intern. Observation of the Intern over the course of their shift will be used to determine if the Intern can prioritize reference chat in relation to other Intern responsibilities.
Evaluation - If the Interns demonstrate that they can properly prioritize chat questions when receiving inquiries from multiple sources, he/she will be checked off on that particular objective. If not, they will receive an additional training session to help them with the skill, and they will be assessed again the following week. This will continue until the skill is mastered.

This is an authentic assessment that will be given informally to individual Interns. The assessment will be given at the reference desk computer where the interns work.

Goal 5 - The Intern should be comfortable using the reference chat services.
Objective - Interns will demonstrate that they are comfortable using the reference chat services.
Assessment - Because there is no way to really determine if a person is comfortable performing a task through observation, this objective will be assessed by means of a discussion between the instructor and the Intern. The Intern will be asked a series of questions in order to determine if they are truely comfortable using the chat services.
Evaluation - If the Interns relate that they are not comfortable with any aspect of the reference chat services, an effort will be made to help them with issues that they have. After these efforts have been made and the Intern has been given time to impliment suggestions made by the instructor, another discussion will take place. This will continue until the Intern is truly comfortable using the reference chat services.

This is an informal, alternative assessment.

Thursday, November 15, 2007

Instructional Strategies

Type of instructional strategy used for this training - Procedure Strategy

GOAL:
Reference Department Interns will receive sufficient training in three weeks to enable them to use the Reference chat service confidently and in such a way as to achieve 100% patron satisfaction.

KNOWLEDGE STRUCTURE:

Objective: Interns will demonstrate that they can log into both Trillian and Velaro chat software using the computer at the Reference desk.

Trillian is a full feature chat program that imports and supports other instant messaging programs like Yahoo, ICQ, MSN, IRC and AOL instant messenger AIM as well as many others. Velaro is for those patrons who do not have other instant messenger accounts. The two chat services serve the same function, they simply give the patrons an option in regard to access. Both services can accessed by patrons through the library’s website.

STEP: The Intern should be able to recognize the icons for each chat service and be able to log onto each at the start of their shift.

STEP: Volume controls should be checked to make sure that they are on. This ensures that the Intern or Librarian will hear when someone asks for help and they are busy doing something else.

Objective: Interns will demonstrate proficiency using the different tools available to them when using the Reference chat services.

STEP: The Intern should be able to use the Page View feature. This enables them to view the page that the patron is on in real time.
EXAMPLE – Patrons often request help because they are having problems locating the information that they need using the library website, and they often have trouble explaining where they are in the website. By using this feature, it is possible for the Intern or Librarian to view what is on the patron’s screen and therefore provide them faster and more accurate service.

STEP: The Intern should be able to use the Push Page feature. This allows them to push a page directly to the patron. If the patron has a pop-up blocker, the push page is sent as a link.
EXAMPLE – The push page feature is another time saver. It cuts out one step in the process of information transfer by opening the page on the patrons computer screen without them having to click on a link. It can be a little disconcerting when this happens, however, if you’re not expecting it, so it is recommended that you warn the patron that you may be doing this up front.

STEP: The Intern should be able to use the Transfer Files to Customers feature. This allows them to send documents, files, etc. to the patron while you are chatting with them.
EXAMPLE – This is not a feature that is often used during the course of Reference chat, but it can be very useful in some circumstances. If the patron requests information that is not available online, it can be sent using this feature.

Objective: Interns will demonstrate that they are well-versed in Reference chat etiquette.

STEP: The Intern should be able to keep the patron up to date on their progress at regular intervals when it is taking them a while to find the requested information.
EXAMPLE – If you are in the middle of a chat reference interaction, and you are searching for articles in the library databases, but it is taking you a long time, it is customary to send a brief message stating something to the effect of, “I’m still looking,” just so they know that you haven’t forgotten about them. Similarly, if a patron comes up to the desk while you are in the middle of a chat reference interaction, it is a good idea to send them a message telling them that it is getting busy, but you are still working on their problem. It just might take you a little longer. Generally, patrons are very patient and willing to wait.

STEP: The Intern should never tell a patron what to do in order to find the information they are looking for. Rather, they should suggest actions that the patron can take to find this information.
Example – When an Intern or Reference Librarian helps a patron with a problem via reference chat, it is easy to fall into the habit of telling patrons what they should do. Especially when it is busy. If an Intern finds an article in Academic OneFile, for example, they should not tell the patron to go to the database list on the library website and find Academic OneFile and enter a particular search. Rather, they should say something to the effect of, “I have found an article that I believe will help you, if you follow these simple instructions, I can help you get there.” It is even better sometimes to provide links that the patron can follow without actually having to walk them through the process of finding the article, but it is important to teach patrons how to use the library resources as well.

STEP: The Intern should be able to react in a calm and reassuring manner when patrons become angry or difficult.
EXAMPLE – Patrons often wait to the last minute to contact the Reference staff with questions. By this time they are frustrated and in a panic. This sometimes, though not always, results in short tempers on the part of the patrons. The key here is to keep it on the part of the patrons! It is important to be prepared for this eventuality and to remain calm. Usually patrons calm right down when they see that you are really interested in helping them. If they refuse to calm down then you need to contact a Librarian on duty or in extreme cases, Security. Regardless of the outcome, it is important that you remain calm. Becoming agitated yourself will only exacerbate the problem.

Objective: Interns will be able to prioritize Reference chat questions in relation to other Intern responsibilities.

STEP: The Intern should be able to handle questions that come from multiple sources at one time and understand where Reference chat falls into the scheme of prioritization.
EXAMPLE - The standard procedure is to work with patrons who are physically present first. They are the priority since they are actually in your presence. If someone phones with a question and you are busy, you take their name and phone number and tell them that you will call them back as soon as you are able. With Reference chat, if you are busy when a patron contacts you via chat, you just tell them that you are in the middle of a Reference transaction and will be with them as soon as you are able. It is customary to give them occasional messages to let them know you are still there and that you haven’t forgotten them. Sometimes they elect to leave the conversation, but usually they are happy to wait.

Objective: Interns will be comfortable using the Reference Department chat services.

STEP: Interns will practice using the chat services over the course of this training and they will use it under the supervision of a Librarian for a period of time before being expected to use the services on their own. This should ensure that they are comfortable using the chat services.

PRESENTATION:
This training will have several components. The first component will be a presentation and demonstration of the Knowledge Structure discussed above, and will be accomplished using a combination of PowerPoint and demonstration of procedures using the computer and overhead projector. A PowerPoint presentation will be used the first day to lay out the knowledge structure of the training. Classroom discussion will be incorporated into this presentation. This will take one day. The next component will be a demonstration of each particular task and accompanying steps. The demonstration will be based on actual chat interactions that have taken place here at Jackson Library. A Librarian downstairs in the Reference Department will send inquiries to the instructor's computer in the training room using both Velaro and Trillian. The instructor's computer screen will be displayed to the class with an overhead projector. I will respond in the manner in which you, as Interns, will be expected to respond, and we will discuss why I am responding as I am. This, too, will take one of our training days.

EXPLORATION:
After the presentation phase of the training has been completed, you will each have an opportunity to play out chat scenarios based on real-life chat interactions. Each intern will come up to the front computer and participate in several mock Reference chat interactions. This will be identical to my demonstration, except that it will be you up here at the computer, instead of me. At some point every one of the tasks and steps will come into play, but every individual may not personally handle each task or step. Your classmates will be able to see the transaction as it occurs on the overhead projector. If you feel that you need assistance, ask the class for suggestions. The point of this exercise is to become more at ease with the system itself and each particular task in particular. There is no grade involved here. It is just practice, so do not be afraid to ask for assistance. If I feel the need to provide assistance, I will do so. This will take two additional training days.

PRACTICE:
The practice will occur during the first two weeks of actual reference duty. Each of you will be on duty with a Reference Librarian. You will handle all reference chat questions, but the Librarian will be there to oversee the interaction and provide assistance if necessary. This will take place primarily at the reference desk computer. Intrinsic feedback will come from the patrons themselves, while the extrinsic feedback will be provided by the Librarians. After this two week period, you may or may not be on duty with a Librarian. You may, of course, continue to ask for assistance if needed, but you will be in a more responsible position and you may often find yourselves providing reference chat service when there is no Librarian around to assist you.

LEARNER GUIDANCE:
The Learner Guidance will occur as the Librarian gives assistance to all of you during the two week practical training period. We will meet at the end of this two week period and discuss any issues or questions that you have about your experience so far. I will print out some of your chat interactions and we will discuss them, without disclosing which Intern actually handled the interaction. We will talk about what was done right and what could have been done better. This will give you a chance to ask questions that you perhaps wouldn’t have thought of before gaining practical experience.

The last thing you will be asked to do at the end of this training program will be to give your feedback about the training process itself. Because this will occur after you have been working on the desk for two weeks, you will be in an excellent position to comment on the effectiveness of the training, and you will be afforded the opportunity to give suggestions about ways in which you believe the training can be made more effective.