Revised APPR Assessments Part 2: 1st Grade

In my last post, I told you about my revised assessment for Kindergarten, which is serving this year as my LLO portion of my APPR.  In this post, I'll share how I changed the rubric for my 1st grade assessment and why.

Last year, 1st grade served as part of my SLO and while the assessment itself was a great thing, the rubric I had created was just too simple and hard to get a good, accurate measurement of growth out of.  You can see the post I did at the end of last year here.   This assessment involved asking the students to draw 6 things in 6 squares.  They were to be assessed on the subject itself, use of a horizon line, use of colors, and craftsmanship, but because of the way the rubric read, it was very difficult for the other art teacher who assessed my tests to give an accurate grade.  For one drawing, a student might have received a 4 for one aspect of their drawing but a 1 for another, which made it difficult to give a solid score for each drawing.

It was also hard to assess the "draw anything you want" square because let's face it, a lot of the 1st graders just drew "scribbled abstract art".  In the end, I didn't meet my 80% mark of students who met or passed my goal...I only had about 65% of students meet or exceed the goal.  So, here is how I changed this assessment for this year.

The first thing I did was change the six items students need to draw.  Now, they have to draw a fuzzy dog, a rainbow, a tree, a landscape, a self-portrait, and a still life.  I like the change in the last three options because now they have to prove that they learned content as to what those three subject types are.  I also changed my rubric.  It's a little bit more complex now, but I think it will be a lot easier to grade these drawings and get a more accurate show of growth.  

1^st Grade: 6-Square Drawing Test

Fuzzy Dog	Texture & Color    Student used appropriate colors for the subject and details drawn.  Student made object look like it has the appropriate texture.	4        3       2       1       0
	Dog Body Dog body is drawn accurately and looks like an actual dog.	4        3       2       1       0
	Background Student included appropriate details in the background for the subject, including a horizon line.  Sky is colored all the way to the horizon line.	4        3       2       1       0
Tree	Texture & Color Student used appropriate colors for the subject and details drawn.  Student made object look like it has the appropriate texture.	4        3       2       1       0
	Tree Structure Tree contains all appropriate parts (trunk, branches, leaves and possibly roots).  “Cloud tree”=3, “ice cream tree”=2, “lollipop tree”=1.  No tree=0.	4        3       2       1       0
	Background Student included appropriate details for the subject in the background, including a horizon line. Sky is colored all the way to the horizon line.	4        3       2       1       0
Rainbow	Color Order Correct colors ROYGBIV & order = 4 Correct colors ROYGBV & order = 3 (no indigo) Correct colors but not in order (or missing 2 colors) = 2 Did not use correct colors but has 3 colors present = 1 Used 1-2 colors (or no rainbow) = 0	4        3       2       1       0
	Background Student included appropriate details for the subject in the background. Sky is colored all the way to the horizon line (if one is included).	4        3       2       1       0
	Craftsmanship Student used neat coloring.  Colored the entire square.  Sky is colored all the way to the horizon line; ground is colored from the horizon line down to the edge of the square.	4        3       2       1       0
Landscape	Type of Art Student correctly drew a landscape.  Included images from outside where the nature is more important than man-made objects.  (Mountains, sky, sun, grass, weather, animals, etc.)	4        3       2       1       0
	Details Used appropriate colors for the subject and details shown.	4        3       2       1       0
	Craftsmanship Student used neat coloring and colored everything.	4        3       2       1       0
Still Life	Type of Art Student correctly drew a still life.  Drawing is of inanimate objects such as a vase of flowers, plants,  fruit, cups, dishes, etc.	4        3       2       1       0
	Details Used appropriate colors for the subject and details shown.  Included a horizon line.	4        3       2       1       0
	Craftsmanship Student used neat coloring and colored everything.	4        3       2       1       0
Self-Portrait	Type of Art Student correctly drew a self-portrait.  Student drew a picture of him or herself.	4        3       2       1       0
	Details Used appropriate colors for the subject and details shown.  Included all appropriate facial features (eyes, nose, mouth, ears, hair, eye brows, freckles, glasses, etc.).  If a body is included, student has shown all appropriate limbs (neck, arms, legs, feet, hands, torso, etc.).	4        3       2       1       0
	Craftsmanship Student used neat coloring and colored everything.	4        3       2       1       0

4=Advanced                  3=Satisfactory             2=Improving         1=Needs Improvement            0=Incomplete

________________ / 72 points = __________________ %

Revised APPR Assessments Part 1: Kindergarten

I didn't really share the outcome of my post-assessments last year, mainly due to the fact that there was some confusion around them with calculations.  In the end, I "passed" and did all right, but my results weren't exactly where I wanted them to be for the post-assessment scores.  Thus, I revamped two of my assessments.  Last year, I did my SLO assessments in 5th, 2nd, 1st and Kindergarten.  My LLO was in 4th grade.  Lucky for me, I have one less to do this year!  This year, my SLO assessments are with Studio Art, Art 7, and 1st grade and I chose to use Kindergarten for my LLO so I could revamp my rubrics.

Today is our first Staff Development day and luckily, I have no place to be except in my classrooms working on my SLOs and LLO, which are technically due today!  In the next few posts, I'm going to share with you my old assessments from last year and what I plan on doing differently this year.  I'll also share with you how I am setting my goals, as I am doing something completely different this year as well.

Last year I did a cutting and gluing rubric and I had a 97% success rate at students passing, which was great, but made it seem as though the assessment wasn't quite as rigorous as it should have been.  So, this year, I changed up my plans for the Kindergarten assessment, and I've actually changed my mind twice about the pre-assessment.  Back in September I shared a post with my ideas for this year, which you can see here.  I originally gave students a shape sheet with fewer shapes and then a quiz that quizzed them on shapes, lines and basic color theory.  I was struggling to figure out how I would make sure those kinders could read their color words by the end of the year to take a "quiz" (since I would essentially have to create a test bank), and then I decided to scratch that idea, after I had already given out that assessment.

Now, I've decided to continue using the shape sheet for cutting and gluing but also the Hey Diddle Diddle Shape Cow project we just did.  This way, I am still assessing their cutting and gluing skills, but also their ability to draw shapes, their use of crayons and how they color.  As I looked at all of these projects, I realized that even though I was trying to teach them how to use the shapes and how to color properly, it's going to take a while for them to improve, so I should still be able to show growth.

  

These are the two activities being used for the kindergarten pre-assessment...

And these are the two rubrics being used for the kindergarten pre-assessments...

At the end of the year, students will then be assessed on cutting & gluing, as well as their coloring and shape skills all in one project...the primary playground project!  In that project, students have to cut out their own shapes and create a playground.  It will assess their ability to cut out shapes that are discernible from one another, how well they glue the shapes down (still using the dot glue method), and how well they utilize their entire paper.  Last year I did this as a review projects for primary colors and shapes, but it'll make a great end of the year assessment, I think.  Instead of having the students do it on 9"x12" paper, I'll give them the bigger sheets like I did for their pre-assessment cows.

 

Two kindergarten examples from last year, done back in February.

And this is the revamped rubric for assessing the post-assessment.

I know these rubrics are hard to read...I'm doing this at school right now and don't have a nicer way of taking a screen shot, so you can find them on the SmARTteacher.

Some more DDI...(text heavy post...and not so much about art :-/ )

I can honestly say that I can't wait for September to be over!  There is just SO much paperwork that needs to be done and turned in, too much thinking I have to do to plan ahead for DDI, SLOs, LLOs, etc. that I think my brain is already fried, and it's only Tuesday!!!  (Boy, I'm sure missing have day 6 off at the moment...this being a full time teacher is going to wear me out! ;) )

Monday I stayed for a faculty meeting about DDI, data driven instruction.  The presenter came from our local Boces and has been working with other districts to begin implementing DDI as well.  The information she presented does make sense.  The first year of implementation is going to be a little more work because of the need to build test banks and such, but the concept behind DDI makes sense.  I figured I would just share a bit in this post about what was presented and how I think I'll use it with my high school art classes.

Data driven instruction is built upon four steps.  The concept comes from Paul Bambrick-Santoyo and his book Driven by Data.

1.  Assessment:  Teachers should be using interim assessments (my district is requiring us to do a test at the 10, 20, 30 and 40 week period...the 40 week being the final exam, and also in my case, part of my post-assessment for my APPR) throughout the school year to see what students have a good understanding on and to see what they are retaining.  Using a ticket out the door (or exit slip) and bell ringers are also forms of assessment that should be used to see what students are understanding.

2.  Analysis:  The second step is to analyse the data from the tests.  Ideally, if you're in a large district, you'd have a team of teachers who teach the same subject area that you can sit down with, or, if you're like my district, my principal and I will be a "team" to analyse my data.  As you look at the data, you should be saying, "Why did 70% of the students choose this wrong answer over the correct answer?" Well, maybe it was a poorly worded question on the test, or maybe the concept needs to be retaught.  On the other hand, you would hope to find that 90% or more of students would be getting correct answers.  (This is where eDoctrina will come in handy that I have to use.  Even though it's a lot of work to input questions, the analysis portion becomes easier because the program will automatically calculate the percentage of student answers for each question.  It even creates a bar graph for those people who need it visually!)

The presenter gave another example of how teachers in another district who have already implemented DDI last year analyse the data.  After a test has been corrected, they assign the following as homework:  "Go home and for each answer you got wrong, please write down and tell me why you chose that answer over the correct one."  This would essentially help you understand how the students heard the information that you taught.

3.  Action:  The third step is to take action.  Meaning, what and when are you going to reteach, and how are you going to reteach it so that students grasp the knowledge better?  Basically, it means you might need to change your teaching strategy.  This step is often the hardest step, especially for those on a time constraint because of state testing.  I think this part is a lot easier for us special areas to do because we generally don't have a state curriculum we are mandated to follow.

4.  Culture:  The final step has to do with changing the culture of student learning.  Basically, under DDI, you "own" your lessons for the first 10 weeks of school.  After that 1st interim assessment, the students "own" your lessons and drive the teaching and learning process...they essentially take charge of their own learning.


So, here's the first question this brings up...isn't DDI suggesting that we "teach to the test", which is what everyone hates doing?  Not necessarily.  Those interim assessments aren't supposed to be held against the students.  But then how are we supposed to get the students to take them seriously and actually try on these tests?  That's up to the teacher.  Our district suggested taking the interim tests as a homework grade instead of a test grade so that they are still held accountable, but it isn't going to make or break their report card averages.

Next, how should we be building our assessments, especially if we have extremely high achievers and low achievers in the same class?  The presenter suggested building the assessment for the bulk of the students.  (Hmmm, easier said than done?)

How many questions should be on these interim assessments?  The presenter suggested that for every standard or sub-standard you have covered up until that point, you should have 3-5 questions per standard.  (In eDoctrina, when I input my questions for the question bank, I can also attach a standard/sub-standard to each question.  Again, when it's time to analyse my data, it will automatically tell me the percentage of students that are doing well/poorly on each standard.  I plan on inputting all my questions from each unit test I give so that I can randomly pick a few from each unit for each interim assessment.)

Coming from the CSE angle... how are we supposed to accommodate everyone who needs it? My district has A LOT of students who require services from an IEP or a 504 plan...extended time, test read out loud, etc.  On top of regular class tests, quizzes and assignments, this just adds that much more stress to those services.  According to the presenter, if the accommodations can happen at the summative, it should happen at the interim.  For example, in NY, a math exam for the 3rd-8th grade is allowed to be read out loud to those who have that requirement on their IEP, but certain sections on the ELA are not allowed to be read out loud.  (This is not necessarily going to be a good thing for many districts...especially districts like mine that can't afford to hire more support staff due to the lower scores by the new state tests under Common Core.  My district is going to try and entice students to stay after for AIS services after school.  This would ensure they get the extra support they need without filling up the school day even more.  Our teachers would be paid extra for staying after to work with the students, which is cheaper than hiring more full-time staff.  Only problem is...students aren't mandated to take AIS, so we have to make them want to stay after.)

Wow, so all that being said, DDI is looking good and bad to me right now.  To be honest, checking student understanding at various points using cumulative assessments seems like a no-brainer to me.  I did this last year with my 5th graders and my color theory curriculum.  Each rubric had review questions from all of the previous lessons and units, and each rubric, there were more and more, but they were able to answer just about all of the correctly by the end of the year!  (Thus the reason they did really well on their post-assessment!)  I remember tests from high school that had review questions/bonus questions on them from previous units...

DDI does stink though, because now I'm having to administer more tests in art.  I do think tests are important because I want to treat art, to an extent, like an academic subject so that students take it seriously and actually try to learn and maintain the information I teach them.  But holy tests galore!  I'll have to work on getting a little creative with these interim assessments so that they have some performance based aspect to them...but that might be too much for my plate this year, I'd want to give it serious thought!

I just want to leave you with one more thing.   (And I promise, after this post, I WILL be posting some artwork and such that is starting to be finished up at school!  I still have only seen about half of my elementary kids...!) The presenter gave us this handout to try and explain why DDI needs to be done (and it comes from Bambrick's book):

TEACHER:  Listen; this data-driven education thing seems interesting and all, but why are we doing it?

PRINCIPAL:  Do you watch basketball?

TEACHER:  Sure.

PRINCIPAL:  During a recent high school basketball playoff game, the scoreboard completely malfunctioned midway through the game.  So, the refs kept the score and time on the sidelines.  As it came close to the end of the game, the visiting team was down by two points, but they did not realize it nor how much time was left.  The clock ran out before they took the final shot.

TEACHER:  That's not right!

PRINCIPAL:  Of course not.  If the scoreboard had been working, the entire end of the game could have been different.  So you'd agree that a working scoreboard is critical for sport events, correct?

TEACHER:  Of course.

PRINCIPAL:  At the end of the day, data-drive instruction is like fixing the broken scoreboard.  Relying on state tests is like covering up the scoreboard at the beginning of the game and then uncovering it at the end of the game to see if you won.  At that point, there's nothing you can do to change the outcome!  We use interim assessments to keep the scoreboard uncovered, so we can make the necessary adjustments to be able to win the game.

Ugh...Data Driven Instruction???

Well, today I was introduced to the realm of data driven instruction and if I haven't felt overwhelmed by this point, I sure do now!  Anyone else out there having to deal with DDI?  I feel like I had it easy last year with my elementary SLO's compared to this year!  Luckily, it looks like I may only have to do 2 SLO's and 1 LLO this year (compared to 4 SLO's and 1 LLO last year), but it's going to be SOOOO much more work to do them with this data driven instruction component!

For anyone who doesn't know what DDI is, from what I learned today, teachers must give students an exam-like assessment at various points throughout the year (our school is making us do one at each 10 week quarter to assess what has been taught up to that point).  We have to review what a majority of students are getting wrong/struggling with and readdress/reteach, and then test again at the next 10 week period!

Now, I get the idea behind this, but it's going to be A LOT of work for me!  We are now moving to using eDoctrina (which I found out is a relatively locally ran company in Buffalo, NY...near my hometown and 5 hours away from where I teach).  In eDoctrina, we will enter in our SLOs, pre-assessment scores, target scores for each student, and then the post-assessment scores.  The nice thing is that eDoctrina will automatically weigh and compute our 40% assessment score for APPR. (On a side rant, I'm being encouraged to use my student's STAR scores for my SLO's...something I DON'T want to do!  I'd rather do a two-part assessment...an observational drawing portion and a test portion that tests students on art vocabulary as well as a writing portion, that will probably be analysis of an artwork...but that's another post for the future!)

The other nice thing is that eDoctrina has test banks to build test (for the quarterly DDI exams) as well as for post-assessment tests IN EVERYTHING BUT ART! (Because there is no state assessment for art, and no state assessment for art from other states that I've been able to find elsewhere so far...)  Which therefore means that I will have to start entering in test bank questions!  Argh!  It will be worth it in the long run, but it's so daunting to look at right now! This program will generate tests and bubble sheets to match the test.  After the bubble sheets are filled out, you scan the bubble sheet (which is printed off with a special bar code) into a fax machine to the eDoctrina number and the results are automatically calculated and entered under each specific student's account....thus allowing the program to generate graphs and data for me as a teacher to evaluate and adjust my teaching.

There is so much more I could say about this program, but for now, I'll just ask if anyone else out there is venturing into the realm of DDI and what your thoughts are on it...how is your district approaching the data collection...and are you using a program like eDoctrina?

4th Grade LLO Results

For the local portion of my "grade", I used my 4th graders.  Again, as with the 1st grade, I will opt to give individual goals for each student rather than saying "80% will reach...".  It's easy to see that EVERYONE definitely improved but I ended up falling short with the number of students reaching the goal.  In a lot of case, students were only 1-3 points off from reaching my goal!  Oh well.

For this assessment, I had students do a still-life drawing from a white still life.  At the beginning of the year, I gave them two days.  Everyone finished well within that time frame.  At the end of the year, they received 3-4 days.  Even the early finishers needed at least 2 1/2 days to do theirs, so this made me happy that they weren't rushing!

For the final assessment, students could shade their still life with pencil or they could paint it with acrylic, depending on which they preferred.  All year long we did a variety of projects to reinforce using value, such as the value cities, pastel poinsettias, the fall drawings with leaf prints, and the still life practice just before our post-assessment.  We also did a lot of drawing from real life this year to reinforce observational skills.

Here are the results!  Pre-assessments on the left, post-assessments on the right!

(Please pardon the junky photos of the pre-assessments...those were done at the beginning of the year before I got my new camera!)

Drastic improvement, just in this student's willingness to participate!

Great improvement in shading skills...as well as this student's self-esteem in art.  It's always amazing once a student can do once they realize their confidence!

My private lesson student.  She lost some of her value in the foreground bottles, but none the less, did an AWESOME job mixing paint on the canvas.

Awesome value improvement.

1st Grade Post-Assessment Results

For this particular group of students, my SLO stated that 80% of students would reach a satisfactory grade on their post-assessment (which was a range of 72%-80%).  Most of the students scored below a 65% at the beginning of the year on their pre-assessment.  Unfortunately, I did not reach my goal with this SLO, despite the fact that each student did in fact improve.

I have definitely learned a bit about writing my SLOs for next year.  One thing I learned is that I think it may be more realistic to set individual goals for each student, and then state that 80% of the students will reach their goal.  I also realized that I need to make sure I take the time to clip together each individual pre- and post-assessment.  I didn't have time to do that before I swapped with the other art teacher, and she did not take the time to go back and look for the pre-assessments to make a comparison to.  Therefore, some of her scores were probably a little lower than what I would have given the students.

In the end, I'm not worried about how these SLOs will affect my final score for APPR.  Since I had to do 4 SLOs (and other teachers only had to do 1 or 2), mine will all be weighted differently for that portion of my score.  And also... this is just a test year for our district!  I've learned from my mistakes and now I can fix them in order to succeed better next year!

So, without further ado, here are some of the examples of the pre- and post-assessments from 1st grade.  Students had to do a 6-square drawing test.  They had 5 minutes for each square to draw the subject given, show good craftsmanship and a good composition (meaning, have a horizon line, sky colored to the ground, good details, fill the square, etc.).  Students had to draw a rainbow, a fuzzy dog, a flower, a tree, their family and anything.  These were a little hard to grade using a 24-point rubric (4 points per square) so I think if I use this assessment next year, I will make more of a checklist for the drawings (i.e. a check if the dog looks fuzzy, a check if their is a horizon line, a check if the sky is colored all the way down to the ground, etc.).

Pre-assessment on the left, post-assessment on the right.

This student improved DRASTICALLY this year.  This student is very much behind the others maturity wise and also cognitively due to a health issue when younger. It's so great to see them draw people with bodies instead of arms and legs coming out of the head!

Nice improvement with color choice...this student is starting to recognize specific color choices instead of using one color.

Love seeing the sky colored all the way to the horizon line...and also when they stop writing words with arrows above their subjects!

Great improvement!  People are wearing clothes!

Love the details on the tree and rainbow for this one!

Kindergarten Post Assessment Results

I am happy to say that I reached my 80% goal in the growth department for my kindergarten SLO!  My goal was stated something like this: "80% of students (39) will reach a score in the satisfactory grade level (72%-80%)."  I did low-ball it for the grade level that they would reach, but that's what we were told to do.  I almost think that next year I'd rather state my SLO's more like this, "80% of students will score 6 points more on their post-assessment."

That being said, I will share that I was successful on my 5th grade SLO at reaching my goal as well.  Only about 65% of my students reached the goal on the 1st grade SLO, and less than that on my 2nd grade SLO.  After discussing it with the art teacher who graded my assessments, we both think it's because the scavenger hunt was just too difficult of a task for their level.  My 1st grade rubric for the 6-square drawing assignment also needs to be revamped into more of a check-list.  It was hard to grade the post-assessments based on what the rubric said because many students had great horizon lines and backgrounds, but weren't as successful with the subject matter.  My rubric dealt more with the subject matter and didn't allow for points to be scored for the craftsmanship in each square, if there was any.  But more on that SLO in a later post.

I posted about my SLO's when I started blogging back in July...you can see my original post here.  I did have to change my grading rubric due to the changes in how our assessments had to be graded at the end of the year.  Since I had to swap assessments with another art teacher, presumably one who does not work in the district and would not be there to observe student's cutting skills, I had to take out any observational categories in the rubric (such as students knowing what hand to hold their scissors in).  Here is my updated rubric that I used for the pre- and post-assessment.

The following are examples of student's pre- and post-assessments.  The pre-s are on the left and the post-s are on the right.

Improvement at being more precise.

             

Improvement with composition as well as preciseness at cutting on the lines.

Improvement at cutting on the lines more precisely.

Another improvement with speed and also the length of the snips with scissors.

This student had really great control over the scissors at the beginning of the year, but took too long to cut things out!  She always needed an extra day on projects in order to finish them, but she was able to finish her post-assessment in one day!