The El Sistema-inspired field is filled with questions. How could it not be, given its recent and rapid emergence? Our challenge, then, is to find the right questions – the provocative, testing, and ultimately, formative ones.
One of the focal investigations of the Sistema Fellows this year was how to document and measure the effectiveness and impact of El Sistema-inspired programs in the United States; that is, carrying out evaluation and assessment. The terms “assessment” and “evaluation,” while highly related, refer to substantively different things. Assessment measures the student learning that takes place, whereas evaluation measures the quality of the learning experience (or programming).
Our exploration of these topics proved much richer and more robust than we originally expected once we realized how many facets there were to these topics, from both social and musical standpoints. However, three key overarching questions emerged from our process:
For whom do we assess?
What do we assess?
How do we assess?
As a field, we often feel the need to prove our program’s value to those on the outside through our evaluation and assessment results; assessment, however, can and should be just as introspective as outward looking. We advocate using assessment tools and practices for the purpose of determining effective programming and teaching, improving students’ learning and experiences, and ultimately leading to longer-term outcomes and impact. Thorough and intentional assessment has the potential to inform program growth and provide a clear direction for the field.
Continue reading by downloading our document.
Andrea Landin, Carlos Roldan, Elaine Sandoval, Sara Zanussi
Sistema Fellows '13
Nice paper! Kudos! It's great to use both qualitative and quantitative methods of measurement (assessment). One requires lots of preparation out front (quaNtitative) and the other on the back end (quaLitative). That is, if they are to be done properly. I used both extensively in my PhD dissertation related to music aptitude and achievement.
Music literacy, to me, infers reading and writing, much like language literacy. You usually don't say a 3 year old if very literate, though they think and can speak eloquently at times. Music literacy should be built upon a large foundation of listening, and performing skills, PRIOR to introducing literacy skills—reading and writing. If the foundation is not in place, learning will be haphazard for most of the children. Rating scales for performance achievement are key is determining objective progress in the child's ability to understand the tonal, rhythmic and expressive dimensions of music. Technical skills can be measured as well this way. Creative and improvisation skills can also be measured though most have difficulty developing appropriate measures and providing the children with the readiness to be successful at these "tougher nuts to crack."
The key point you make is valid: for whom, why, and how do you do the assessment? For me, I believe strongly that measurement is for one overriding purpose—for me to improve my instruction. For the child, a test should be something from which to learn, not just a marker of progress.
All would do well to read Edwin Gordon's chapter on Measurement and Evaluation in Music Learning Sequences. giamusic.com The reliability of our measures needs to be high enough to trust. Otherwise, you're judging on a subjective basis and this, by its very nature, cannot be reliable. Anything unreliable cannot be valid. This is the science of statistics.
Say I got a 34% on a test. What grade should I get? The number of items correct on the test is measurement—a ruler, albeit a bit stretchy in most cases. The grade I should get is a subjective determination which should be based on both normative (within the group I'm being measured against) and idiographic (me being rated against myself [my aptitude, for instance]) factors. Sorry for the bad sentence. What if 34% was the highest score of all on a test written for 12th graders and I was in 4th grade. That's more than an A in my book. What if 34% were questions on a True-False quiz. A monkey should have done better, right? Fail for sure. This is evaluation.
A child with high music aptitude should be held to an objectively higher standard than his peers. A child with low aptitude should be nurtured along while the high achieving student can be asked to keep stretching himself further and faster. We are not all equal in our abilities to learn music. This is a no-brainer. And yet, we teachers tend to treat each child similarly, as if they were a homogeneous group. Think about it for a minute. It's tough to confront, but I would be willing to bet my house that almost all of use don't know or understand the objective tonal and rhythmic music aptitudes of our students. How do you know who to push and who to nurture? A teacher who goes on prior performance instead of a valid aptitude measure is likely at cause in the moderate to severe underachievement of 40-50% of his children. This is harsh, but it's borne out in the research.
When most of our children come close to reaching their potentials, it will be a big day for music education. Typically, only a handful overcome the obstacle of traditional music education to learn to read, write, improvise, and create music with competence. We teachers persevered through the system because we were both smart and had high aptitude. Most of our students are NOT LIKE US at all. Some are smart, some have high music aptitude, some just love music and practice a lot, and some struggle and drop out. In fact, the dropout rate is 80+% over just a few years. Hopefully, el sistema inspired programs will make a huge difference because of the passion (and hours) we bring to the party. Kids are doing extraordinary things. I just know that we're just scratching the surface when it comes to music achievement based on the above paper and our reluctance to "test" to measure the achievement of our students. It's unconscionable to teach without measuring what the students are learning. It's the BEST way to improve our instruction. For them, it's the BEST way for them to get an objective marker from which they can assess their progress and continue to improve.
There's more to say, but let's leave it there for now. I appreciate you taking the time to read this. Let me know what you think. Some of my work is controversial given the status of higher level music education and it's relatively entrenched ways. Still, el sistema is forging new territory and bringing a new passion to music education. Further, our true el sistema mission is about social change. I think good change will happen as children get hooked into something they are extraordinarily successful at and continue to pass on the passion through several generations of a new breed of musicians that we are helping to create. Hooray for what we do!
Posted by: Dr. RizzRazz | June 23, 2013 at 01:59 PM