1	Some Math Ed “Laboratories” and Related Questions Paul Eakin Department of Mathematics University of Kentucky paul@ms.uky.edu
2	Two avenues to research Select a broad, general question that interests you and focus down to a manageable sub question to study. Select a specific situation or context that interests you, study it intensely, and report on your results Reality is generally a mix Will refer to the second as Laboratory Approach
3	Some Available Research “Laboratories” Large UK math courses AMSP Student programs Explorer Program Excel Program Catsbusters Program Distance learning Professional Development Teacher support Technical Systems
4	Lab A: Large Uniform classes at UK Uniform Examinations Team graded Homework Web-based Syllabus Locally written texts, notes Large (120) and small (35) sections Tutorial assistance programs Mathskeller (CB and library) College algebra 1800 students fall, 900 spring “business calculus” 800 students fall, 1500 spring Discrete math 200 fall, 600 spring
5	Sample questions, contexts Primarily to indicate possibilities Two hours scribbling By no means complete May or may not even fit into proper definition of math education
6	Some questions on UK Courses The math department has for five years given random halves of the KEMTP to the students in MA109 and MA123 on the first day of class. The score, typically out of 12. If p(j) = likelihood of A or B given a score of j then pretty much independent of class the graph of p(j) is a straight line. What is being measured? Would the same result occur for a random selection of problems?
7	"The UK web homework system..." The UK web homework system (and others) is based on software which creates variants problems and versions of assessment instruments. These “change the numbers” in problems, vary the colors and labels in diagrams, etc. In what sense are two such “clones” equivalent? Are there assumptions on the software the will comfortably guarantee equivalence?
8	"The UK online system collects..." The UK online system collects huge amounts of data which can be compared to demographics, preparation, previous courses, etc. It would like to use this information to feed back and improve the course. What does it mean to improve a math course (a service math course)? Are there reliably measurable statistics that can be used for this purpose? One often mentioned concept is “value added”. What is the value that is added? How is it measured?
9	"The web homework system" The web homework system permits the attachment of arbitrary sets of standards and levels to problems and problem sets. This would permit the addition of specialized sets of standards created specifically to investigate focused questions. An example might be a set of bias indices, a priori specification of “difficulty”, “purity”, “real world”, etc. Why do college students do and not do online homework? Are the reasons the same for “paper and pencil” assignments?
10	"Students in the UK courses..." Students in the UK courses all have calculators and depend upon the for the simplest calculations. However, anecdotally they appear to have no skill in using them which relates to an understanding of algebra/calculator syntax. Very few students appear to be able to use the devices for anything other than unary or binary operations although many do long sequences of these. What are the actual calculator skills of the students? It would be possible to use virtual calculators and online homework exercises which capture all keystrokes to actually study this type of question. Are there clear examples of situations where the use of technology unequivocally enhances mathematics instruction (as measured by student understanding)? Anecdotally material involving complex notation, particularly matrices, is significantly enhanced even in small sections.
11	"The UK MA109 and MA123..." The UK MA109 and MA123 courses use faculty-written notes and online support materials rather than commercial texts. These will save UK math students well over $300,000 in AY2005-6. Are there compelling reasons to use commercial texts even in high school? Will UK students retain their $15 sets of notes in higher numbers than commercial texts? Will they subsequently refer to them in other courses? (Since many students take MA109 and then MA123 questions of this type can be studied.) What are the actual textbook economics? What percent of students effectively rent their texts? Has web lookup replaced the need to retain texts for reference?
12	"UK is now using preservice..." UK is now using preservice teachers as assistants in the MA109 program. The structure of the program makes this possible even through they don’t have 18 graduate credits in math. Anecdotally the college teaching experience is excellent preparation for many aspects of high school teaching. Anecdotally the teachers who have come through this program are far more confident than their peers Anecdotally the teachers in this program are much more successful than faculty in retention, student satisfaction, and passing rates Anecdotally the students of faculty do much better in subsequent courses. The implications of features of this program for the preparation of teachers should be studied. It would appear likely that these teachers would be highly competitive, less likely to “burn out”, have easier transition, etc.
13	Lab B: AMSP Student Programs Excel Program (school) Older students assist as in-class assistants Strictly in-class (no remedial or outside tutoring) Students paid $250/semester Teacher paid $100/semester to work with student Teachers write formal proposals and administer programs Explorer Program College analog of Excel Students assist with college instruction Acts as apprentice program for undergraduate teaching assistants Explorers paid $500/semester, UTA paid $1000/sem. Program articulates with teacher prep Pre-service teachers teach in college algebra program (MA109) Own sections, $2500 Program is self-sustaining
14	Questions on Student Programs Anecdotally students in Excel program raise ACT scores significantly relative to peers and a significant percentage indicate plans to go to college and major in STEM and or teaching. Are these true and are they sample bias? Anecdotally the opportunity for young teachers to work with Excel students teachers partially offsets some of the early “burnout” and increases satisfaction and retention. Its this true and if so what are the considerations?
15	"Although the program makes no..." Although the program makes no effort to asses the contribution to the success of the classes in which they work, anecdotally the Excel students contribute strongly both directly and indirectly. Teachers report far less instruction time lost when math-ignorant substitutes replace the teacher. Are these real effects? Anecdotally students who go through the Explorer and UTA programs and continue through to teaching are highly effective and well prepared for (secondary) teaching. Is this an actual effect? Can the program extend to middle and elementary teachers? Which features of the freshman college algebra class are similar and which are different from the high school class devoted to the same material?
16	"Anecdotally" Anecdotally, pre-service teachers in the UTA program are at least as effective as graduate students and more effective than faculty in small sections of college algebra in terms of retention, test preparation, and satisfaction. Also anecdotally the (successful) students of faculty are very significantly more likely to succeed in following math courses. Are these real phenomena? If so are they related to the preparation of the preservice teachers?
17	Lab C: CATSbusters Program Very challenging summer course for algebra teachers with no reference to pedagogy Taught by distance learning with conferencing, web homework, small groups led by graduate students and in-service teachers 2 weeks Participants work at their schools Year-long seminars concentrating on pedagogy Formative testing Online system Teacher/faculty team of materials developers Weekly small seminars with mentors Student assistants (Excel model) Printing allowance Distance support program Surplus computers in classrooms with communications (to be) used for external in-class assistance 11 districts, 30 teachers Pre/post math inventory Teacher and administrator survey Technology assessment
18	Questions on Catsbusters Formative assessment program uses teacher-developed, standards-based instruments in effort to enhance CATS performance. Will it have any effect? Anecdotally the small groups become much more active and productive when teachers are in them from different schools – even middle/high in same district. Is this a temporal phenomenon? Does it have implications for programs to build “learning communities?
19	"Anecdotally the feature of the..." Anecdotally the feature of the pilot program that was most appreciated by teachers was the instructional assistance provided when the development teams visited. Can the use of remote/virtual visitors provide comparable support? Anecdotally the Catsbusters groups have been able to build their 2-hr meetings into their after-school routine and to meet regularly and productively. Informally this indicates that the format is viable for year-long teacher professional development and support programs. Is this actually the case?
20	"There are many program organized..." There are many program organized around the development of materials by groups of teachers in facilitated workshop situations. Anecdotally the results of such workshops tend to mirror items from current texts and released items from testing programs for which the teachers are preparing students. Is this a reproducible, measurable phenomenon? What is the copyright exposure from a typical sample of such work? Standardized tests are often advertised as being largely comprised of such “teacher-generated” material. To what extent is this actually the case in the sense that a comparison is made between the final form of a test and the primary teacher-produced material?
21	Lab D: Technical Systems and Technology The archiving and retrieval of mathematics problems for computer-based delivery requires a formalism for indexing problems. The Mathematics Taxonomy Committee of the Mathematical Sciences Conference Group on Digital Educational Resources some years ago produced a mathematical taxonomy for instruction which is a start. http://people.uncw.edu/hermanr/MathTax/oldtax.htm However there are clearly other parameters needed to produce an indexing scheme. What is a minimal set of attributes which would provide an effective and extensible classification formalism for mathematics instructional materials (including illustrations, software, etc.)
22	"The WHS online system currently..." The WHS online system currently lacks many possible features for lack of people to work on them or properly define them. What are the proper report mechanisms for instruction-related data. For instance the WHS system attach to any item any number of sets of standards and levels and hence will collect all performance information for all students who submit responses to that item in any context. What are the most effective reporting formats for this information for individuals and teachers? The WHS system currently does not offer partial credit. Other than counting the fraction of “correct” responses within a problem are there other partial credit assignment schemes that could be “explained” to the computer and would provide better information?
23	"Technology makes it possible to..." Technology makes it possible to create support materials for parents to assist their students. Is it possible to provide resources for parents who know no mathematics? For instance a parent with no musical ability at all can listen to a child’s practice and even compare it to recorded models of the same piece. Are there comparable strategies for math? Within the WHS system one can create and attach to a problem video clips of someone solving that problem or a similar one. Anecdotally these are quite effective. Also anecdotally they are more effective if at least some of them were created by people known to the student. Is this a real phenomenon? Are the clips actually useful for students?
24	"According to KDE mathematics writing..." According to KDE mathematics writing portfolios do not in general work well because they are all done at the last minute and not over a long period for reflection and improvement. The AMSP News system provides a web-based “blog” environment within which students can enter drafts of portfolio items. The system can time-stamp an archive versions. Lee Roher has piloted this application in Jessamine County. Would the use of such a system effectively address the “procrastination problem” – what automated mechanisms would be effective in prompting students to make progress on their pieces. Are there automated measures of progress?