It is hard to believe it has been over a month since my previous podcast & blog about Technology Standards & Accountability!
It was wonderful to hear from so many of my colleagues and friends - but it was really overwhelming to try and organize a followup to this topic. Since I am relatively new to this blogging/podcasting venue, I am not really sure how best to publish your comments.
I should probably begin with revealing a little bit more about where I stand in this process. It was actually over a year ago that I wrote the core of last month's blog/podcast. I wrote it after I had constructed a web-based, 40 item multiple choice test that we were planning on administering to our middle school's eighth grade students. As I chronicled, writing such a device is very difficult, especially if the items are being written for use beyond your own classroom, beyond your school, and potentially even beyond your school system.
Publishing my thoughts and questions has initiated many conversations, and even created some special opportunities. Earlier this month (February 2006) I spent two days with an expert panel assembled by Florida's Department of Education in the Tampa, FL area. This expert panel was comprised primarily of Pinellas County technology experts (they had written the grant funding the project) and individuals from Florida State University, the
Florida Center for Interactive Media (a non-profit educational multi-media company associated w/FSU), University of South Florida, and Florida Department of Education. This was the second meeting of this group- established to guide the development of benchmarks, standards and an assessment tool for technology literacy. The final product of this grant is a performance-based assessment delivered on the web.
This tool is intended for an eighth grade audience and most of it will involve a series of tasks that take place in window that emulates the type of application typically used for this task. The Florida Center for Interactive Media (FCIM) had designed a similar tool for assessing teacher technology literacy, and the sample I saw was a task conducted in their word processor
invention that they called
WordTech.
WordTech was not really a word processor product, but simply an intertactive screen designed to capture mouse and keyboard interaction as the teacher completed the task depicted in a list of steps to the left of the
psuedo-application.
Consider for a moment what an ambitious effort this is... First recognize that the intent is to deliver this assessment to wide continuum of computers around the state via the World Wide Web. Try to distill common tasks and features of a word processor (or any application necessary for the skills being tested) down to an essential list, then create a graphic user interface that includes these in a platform/application agnostic fashion. Finally program this GUI to function as an application would, and to capture the user's efforts to successfully complete a task in a quantifiable fashion. Although the 40 item test that I had worked on had more than just knowledge-recall types of questions, it was well beyond our school or school district project's scope to create a
psuedo-application to assess performance tasks.
I appreciate this effort as a big improvement over the presumptuous
Microsoft/ISTE partnership-produced
NETS Online Technology Assessment. In assessment tools such as this one, the skill's tested are limited to those performed in Microsoft Office, so it may be interpreted as "technology literacy = Microsoft Product Literacy". Microsoft may be commended on taking some leadership to develop such an assessment, but I would suggest it should be renamed NETS Online Microsoft Product Assessment.

On the other hand, you can see that the development of
psuedo-applications poses a whole new set of problems, and although I was very impressed and pleased with the results of the Florida DOE effort, I was once again reminded of the experience at the science standards project that I mentioned in my last blog / podcast- each of us value knowledge that we have attained and it is hard to let go of that. For instance...I had a hard time letting go of databases...
Growing up with Microsoft Works™ and Appleworks™ - databases were always one of the
BIG THREE business applications in my mind
(word processing, spreadsheets, databases). Later I moved on to Filemaker Pro™ and some flat file and relational db's for web projects. I would never claim to be a database expert, but I still see so much of what we do - managing and accessing information - in terms of databases. Consider how much of current technology is database driven: particularly search engines, shopping carts, inventories, operating system directory information, even our mail clients are (in a sense) a database system.
When Microsoft began marketing it's Office Suite™ (for the Mac first: June 19, 1989 and for Windows in 1990), they added their archetypical presentation software, PowerPoint™, but didn't include a database application. Later on Microsoft Office Professional™ included Access™ database software (MS purchased FoxPro™ database company Fox Software in 1992), but the Small Business edition, Standard edition, & Student/Teacher edition still don't include a database program.
I realize that some of the features of a spreadsheet are similar to a database program - I also realize that Microsoft has transmogrified some of the key functions of a database into
Excel™. But in my mind, spreadsheets are considerably different from databases and databases are quite different from spreadsheets. It is much like comparing a chisel to a flathead screwdriver - you certainly could use either for the other's task, but isn't it better to use the right tool for the task?
But most of those in a Microsoft Office™ world are not exposed
directly to a true database authoring program.
I say directly because as I mentioned before, databases are all around us- even if we are not using a computer, much of our interaction with information, searching, organizing, displaying information is done within the construct of a database paradigm. So now as we are thinking about what eighth grade students should know, the question is one of perception - does a technologically literate person need to have a concept of what a database is and what it does?
Would they benefit from understanding the concept of a database record - a unique set of information made up of fields - information, pictures, data that possibly varies with each record, but is characteristic for each record in the database. Would they benefit from understanding that databases can display this information in a very flexible variety of formats, each layout particularly designed to communicate something different or to facilitate the completion of a particular task. Each layout can hide or display some particular fields that are either necessary or unnecessary for that task. Beyond manipulating the order of records by sorting, won't our students benefit from learning about search fields, filtering data with multiple queries during one search? Maybe we have dealt with that to a degree in standards and assessments regarding web search engines, but I would suggest that standards that are more directly related to understanding the development and use of databases would undergird web search engines standards and help students develop important global knowledge skills as well as the specific technology skills.

Just how pervasive is database knowledge? Shouldn't our students understand the concept of creating choice lists so as to limit responses, and advance their analytical/predictive skills anticipating other's responses. If we are using project-based learning, and a constructivist approach, shouldn't our students be involved in creating their own assessments? Aren't database conventions such as checkboxes, choice lists, and radio buttons a key part of information and technology literacy? I believe a technologically literate person should know that radio buttons imply one choice is accepted, whereas checkboxes (by convention) indicate that multiple answers are accepted.
So now we return to the questions that frame the development of standards-based curriculum and assessment:
We are employing this educational industry buzzword right now:
LITERACY - and in this conversation,we are talking about
Technology, Media and Information LITERACY.
What do we mean by
LITERACY?Is the knowledge/skill essential for our future citizens to communicate and understand their world?
Is the knowlege/skill measurable?
Is the knowlege/skill taught explicitly (that is– does it require a conscious effort to be addressed in the curriculum)?
and probably most key regarding the database knowledge/skill example:
As we identify standards are we trying to determine what is the
current state of knowledge is? — Or should the standards reflect
where we want to move our curriculum?When should the knowledge/skill be taught/assessed?
I did get a considerable number of comments on the last blog / podcast ... many of which were of the ata-boy, keep up the good work nature which I certainly appreciate!
Julene Reed was such a huge encouragement as she responded to my initial podcast invitation:
from Julene Reed, Director of Technology
Palm Educational Technology Coordinator
St. George's Independent Schools • Collierville, TN
Wow! What a great commentary on educational technology and technology literacy. You are right on target with what your thoughts.
This is SUCH a moving target because of the rapid changes in this field as well as the increased knowledge base of our students. I especially agree with the thought that although our students are the "digital natives," they still have much to learn in regard to understanding and skill sets. I fear that we are making assumptions about their overall knowledge base in regard to technology that aren't necessarily correct, and yet I am not in favor of more standardized testing to address this.
----- Julene
I must admit, Julene, I am not thrilled with the idea of more standardized testing either... more on that later!
Early on, I had several colleagues pointed me to technology standards that they were familiar with:
from Dr. Helen Barrett
Researcher and Consultant • Kent, Washington
ISTE developed Technology Standards for students, teachers and administrators.
Their website gives information on how many states have adopted these standards.
----- Helen
from Patsy Lanclos
Smithsonian Laureate
George Lucas Educational Foundation Faculty Associate
Palm Education Training Coordinator/ProviderIn 1997 Texas adopted Standards for Technology Applications and here are the actual standards.
----- Patsy
from Kurt Johnson
Utah State University •
Department of Instructional Technology
State of Utah Core Curriculum Objectives for 3-5, 6-8, and 7-12 are here.
However, they started their development loosely based on ISTE but do not reflect them in the current structure.
----- Kurt
from Paula White
Gifted Resource Teacher
Albemarle County Public Schools •
Crozet Elementary
And Virginia has a technology consortium who is certifying teachers inNETS*T standards. . . It's a pretty rigorous procedure where teachers submitartifacts to show their expertise and are evaluated by trained evaluators.
----- Paula
This was really quite usefuI information, and I while I had referenced the NETS project and recommend that all teachers familiarize themselves with it, I think it is significant that "my state has adopted, or modified these or similar standards."
These comments did cause me to to try and clarify the focus of our discussion– the question I am asking is how much impact is it having? Is there a consistent, conscious effort to develop these skills at the student level in any school system? How is it being fleshed out?
We all know that when a state adopts or creates a set of standards or policies that doesn't automatically change what the students are learning in the classroom... it is a decent first step, but in reality only a first step.
I think that other than with some of the nation's tech savvy superstars, technology skills are not being addressed. Maybe occasionally as the need arises, but is the exception and is not occurring with any scope and sequence such as we see in curriculum mapping in other curriculums (reading, writing, math, science, social studies).
My sense is that these Technology Standards have not gotten the attention that they deserve, and most students have not had formal training in how to use a spreadsheet application, or how to vet the reliability of a website. It is very hit and miss (with the emphasis on miss!)
So we have the Standards, what is the next step?
Here is what I really want to discuss- do we need to develop instructional goals that are tied in to the standards? Should we develop a test? Who should be accountable for teaching (which?) these goals? Do we really want or need another standards based test? Will technology standards get the attention that they deserve without this? Can we justify the high expense of technology in education in today's accountability climate without this? Is technology literacy possible ?
John M. Simi, Technology Specialist
Shelby County Schools • Memphis, TN
We have attempted to do what you are talking about in my district. Two years ago, we developed a set of student technology standards based on the ISTE NETS as well as our own Tennessee state standards. We placed them in a matrix format and suggested at what grade level the various skills should be introduced, developed, and mastered. We also included examples of activities that could be used to help teach the performance indicators included under each standard. The examples we chose are lessons that use that particular technology skill but are taught or used within the context of another curricular area such as science or math for example.
We have been encouraging their use and many of our administrators require that their teachers reference these standards in their lesson plans. I'm not sure we can measure the impact just yet (I love Dr. Barrett's quote from Einstein) but we at least took a stab at giving our teachers some direction about whatthey should be teaching their students in an effort to make them technology literate before they leave our school district. I think this is still a wide open debate and I look forward to following the continued discussion.
If you'd like to see what we've done in my district:
http://www.scs.k12.tn.us/tech_std/default.htm
-- John --
-- I really liked how Mr. Simi's county has organized and created a scope and sequence for the technology standards!
from David F. Warlick
The Landmark Project
Gordon,
I listened to your podcast early this morning and enjoyed it. The music is great, and it is very well produced.
The content, however, leans in directions counter to my current writings and preaching. I'm afraid that I have to hit the road right this minute, but will get back to you as soon as I am settled and have a little more time.
Again, great job!
-- dave --
-- I hope to hear more from David on this, I have been reading his blogs on and off for about a year, and am often inspired. I would really like to get with a few folks with some different views together on Skype or iChat!
from Joseph Morelock, Director of Network & Information Services
Canby School District • Canby, OR
We have been using the NETs standards for a while in our district as well for several years, and have several folks trained in using the ISTE standards. I am going to go out on a limb here...but I think that they are actually quite antiquated in their scope and their focus. They were developed in a time much different than our current reality, and with the convergence of "just-about-everything," I worry that they do not translate well for the future. The tools are really invisible to our students...they use them to communicate and create. We are no longer in the old world of "Web 1.0"- the simple downloading of information that the web revolution started. We are quickly moving toward "Web 2.0," where anybody is an expert ("blogs"), and information is open to change ("Wikipedia"). A simple Google search is not so simple, and may come up with undesirable results for our researching students.
I am pushing us (my district) more toward a pervasive approach to technology use and media understanding, and I have to agree with Gordon that we are not spending enough time on the "Media Literacy" and evaluative practices of available media with students. They are swimming in more information available on their mobile phone than any of us had access to in all of the books we have ever read. We need to help them categorize, evaluate, interpret, and add value to information and the media they find.
Take a look at the Partnership for 21st Century Skills group's work: http://www.21stcenturyskills.org
-- Joseph --
-- This is one of the confusing "melds" of this topic, Joseph– I have heard this topic addressed from a computer-centric, a media-centric, a information-centric, and an applications-centric view. For instance, there is the whole visual literacy viewpoint that seems very important to me. If we are claiming to create a technologically literate populous, is it enough to know how to create a new slide in PowerPoint? Is that really what we mean by technology literacy, or should they have a sense of the general characteristics of an effective PowerPoint slide?
from Dr. Bruce Ahlborn
iTechnology Coordinator
Northbrook Junior High • Northbrook, IL
Let me suggest that what gets tested, gets taught.
While we know better (I think we know better!) our endeavor seems to be focused on coverage, not uncoverage, on misunderstanding, rather than understanding. I think the question to ask would be which local, state and national tests include items that reference any of the standards listed in the NETs project.
Although we know better, our institution is riveted into this infinite loop of testing.
Gordon you are correct, these standards have not received the attention they deserve. I think you should ask about how schools build accountability for integrating technology into the curriculum.
I think for students, the opportunity to engage in learning experiences involving technology, may well be the best motivation for them to show up to school. Think about it, drill and kill, direct instruction, fill in the blank, word search, grinding boredom or engaging in creativity and thinking and problem solving and collaborative projects.
Gordon, sorry for the cynical rant, but then again, you asked.
-- Bruce --
-- Brucie baby, you are dancin' around the fire so nimbly! First you lead with right hook - the unfortunate fact "what gets tested, gets taught" and then you give us the left jab: "our institution is riveted into this infinite loop of testing" And finally I hear the fear that one of the few fun things in education -technology- is about to be codified into another drill and kill lesson (maybe?)
I hope not Brucie, I hope not!
from Dr. Helen Barrett
Researcher and Consultant • Kent, Washington
The ISTE Standards have received that criticism, that they are both outdated and too general, but they were also written to be more timeless, so that they would not have to be rewritten when the next "new thing" was introduced. The student standards not only cover basic operations and concepts, but also social, ethical, and human issues, and tools for productivity, communications, research, problem-solving and decision-making. The teacher standards, which I am much more familiar with, were designed around the teaching, learning and assessment process.
Those who criticize the standards tend to point out specific applications that are not covered that change over time, which is precisely why they were written with such a broad scope. One criticism is also that the broader the competencies, the more difficult they are to assess. ISTE has been working on several books and online assessments to address that issue.
I think the issue is really ACCOUNTABILITY. In schools, we tend to (have to?) teach what is tested, and some states have started moving toward the EETT 8th grade assessment outlined in NCLB. But as schools struggle with the other testing requirements of NCLB, technology is far behind the priorities of reading, writing and math. I hear that schools are under so much pressure to meet "adequate yearly progress" that technology tends to be used in ways that support achieving higher test scores. One thing we rarely asses though, is student engagement.
In this environment, we need more scientifically-based research (SBR) on the impact of technology to support student engagement and achievement. We KNOW it works, but how can we PROVE it? The challenge with this type of SBR is that it is expensive to do, and often provides very narrow results.
What did Einstein say? "Not everything that can be counted counts, and not everything that counts can be counted." ISTE has put up a website on research in educational technology with studies that meet high standards: http://caret.iste.org/
I agree that the Coalition for 21st Century Skills has compiled an impressive set of competencies, and their web site provides a rich set of resources. Underlying their entire spectrum of 21st Century Skills, Tools, Content, Context and Core Subjects is Assessment. Just look at their database of assessment tools, and you will see a lot of tools that can be used for assessing ICT Literacy. Most of them are not free, or work only on Windows, including the one that ISTE developed with Microsoft (a discussion on this list from at least a year ago)! There is also a High School Survey of Student Engagement: http://ceep.indiana.edu/hssse/
-- Helen --
In the middle of Dr. Barrett's eloquent defense of ISTE Standards, is one of the motivations that I have for pushing this discussion out in the open. I fear that statement as schools struggle with the other testing requirements of NCLB, technology is far behind the priorities of reading, writing and math. I hear that schools are under so much pressure to meet "adequate yearly progress" that technology tends to be used in ways that support achieving higher test scores. One thing we rarely asses though, is student engagement.
It really concerns me that I see so many classrooms where technology is only used to send memo's around, print tests from textbook publishers, post grades to the central offices, and most recently (in our district) the big emphasis is on accessing student score data for "informed curriculum decisions". The only time the students get to use the computers is for an electronic worksheet -remediating their skills for our state tests.
So again the question, is the only way to impact the curriculum, to cause administrators to attach value to student technology skills, to get true integration to take place– Is the only way to make us all accountable for technology standards?
from (a former roommate
Jeff Johnson
Director of Technology
Greendale School District • Greendale, WI
Paraphrasing Bernajean Porter:
- If you don't know it's making a difference, why are you doing it (referring to spending scarce funds on technology without having assessments and reporting mechanisms in place that look at the relationship between technology use and learning)
- You get what you get because you do what you do
- Where did anyone get the idea that learning with technology was optional
Bruce hit on the key word: accountability. When a school district decides it's important enough to develop assessments for how technology impacts what happens in the classroom, and hold everyone accountable for meeting standards, change will be more likely to take place.
We've been talking about standards for years but our system of reporting academic progress to parents and to the Board is glaringly absent of references to technology as it applies to curriculum and assessment.
In 2000, Wisconsin developed a document that provides educators with two matrices for looking at how the state's information and technology literacy (ITL) standards relate to the four core content areas. In theory, once teachers see the relationship between their content area standards and the ITL standards (killing two birds with one stone), they'll be more likely to see how technology "fits" and do that. But if it's not important enough to schools to hold teachers and administrators accountable for making that happen, it probably won't.
-- Jeff --
I got so many more comments, there just isn't time to spell them all out... but I really appreciate you all mulling this over with me.
So were quite adamant that the technology standards should be entwined in the regular curriculum. That is the main direction we are going now at our school – each of the MESH subjects are being assigned some domain(s) to cover. At any rate, I sure hope that we are able to give these skills the attention that they deserve! Let's keep the discussion going!