Marcelo Tibau

There’s a song by Leonard Cohen that states “everybody knows” and “that’s how it goes”. The same goes for the fact that the amount of data online activities generate is skyrocketing. This is true because more and more of our commerce, entertainment, and communication are occurring over the Internet and despite concerns about globalization and information accuracy, it’s a trend that is impossible to curb. Like a steamrolling, this data tsunami touches us all, so it’s more than natural that it also catches education. With analytics and data mining experiments in education starting to proliferating, sorting out fact from fiction and identifying research possibilities and practical applications becomes a necessity.

Educational data mining and learning analytics work based on assumption of patterns and prediction. Both disciplines are used to research and build models in several areas that influence online learning systems. The bottom-line here is if we can discern the pattern in the data and make sense of what is going on, we can predict what should come next and take the appropriate action. The business world name it insight and it’s the difference of make “big bucks” or be caught unprepared. So believe me, it’s valuable.

Data mining with educational purposes can be used basically in two big areas. One is user modelling, which encompasses what a learner knows, what a learner’s behavior and motivation are, what the user experience is like, and how satisfied users are with online learning. Well, the same kind of data used to model can be used to profile users. Profiling means grouping similar users into categories using salient characteristics. These categories then can be used to offer experiences to groups of users or to make recommendations individually and proceed adaptations to how an online learning system performs.

A little explanation it’s needed at this point: online learning systems refer to online courses or to learning software or interactive learning environments that use intelligent tutoring systems, virtual labs, or simulations. They may be offered through a learning or course management system and through a learning platform. When online learning systems use data to change in response to student performance, they become adaptive learning environments.

Increasing use of online learning offers some opportunities, such as to integrate assessment and learning and gather information in nearly real time, to improve future instruction. This process goes like this: as students work, the system captures their inputs, collecting evidence of activities, knowledge, and strategy used. Everything counts here, the information each student selects or inputs, the number of attempts the student makes, the allocation of time across parts of the process, and the number of hints and feedback given.

As students can benefit from detailed learning data, so the broader education community can thrive from an interconnected feedback system – such as what works better for a particular content and how to stimulate necessary skills like metacognition. As put by the U.S. Department of Education in a 2010 report (National Education Technology Plan – NETP, 2010a, p. 35): “The goal of creating an interconnected feedback system would be to ensure that key decisions about learning are informed by data and that data are aggregated and made accessible at all levels of the education system for continuous improvement”.

As it’s expected that these learning systems be able to exploit in detail  activity data from learners to recommend what the next activity should be, and also to predict how a particular student will perform in future learning activities, being able to connect the dots and produce insights presents itself as a necessity. It’s precisely here that enters data mining and learning analytics.

Understanding big data

Although using data to enhance decision processes is not new – they are used in what is known as business intelligence or analytics – it’s a relatively new approach concerning education. As their business counterparts, learning analyses can discern historical patterns and trends from data and create models that predict future trends and patterns and comprise applied techniques from computer science, mathematics, and statistics in order to extract usable information from very large datasets.

Usually, data are stored into a structured format, which are easy for computers to manipulate. However, the data gathered from learning platforms have a semantic structure that is difficult to discern computationally without human aid, hence is called unstructured data (e.g. texts or images). To analyze these events is required techniques that work with unstructured text and image data and data from multiple sources. When these data comprise a vast amount, we have the famous big data. It’s important to understand that big data does not have a fixed size, it’s a concept. As any given number assigned to define it would change as computing technology advances to handle more data, big data is defined relative to current capabilities.

Big data, educational data mining and learning analytics

The big amount of data snared from online behavior feeds algorithms and enables them to infer the users’ knowledge, intentions, and interests and to build models that can predict future behavior and interest. In order to achieve this goal data mining and analytics are applied as the fields of educational data mining and learning analytics. Although there is no hard distinction between these two, they have had different research histories and distinct research areas.

In general, educational data mining (also known as EDM) looks for new patterns in data and develops new algorithms and models, using statistics, artificial intelligence, and (of course) data mining to analyze the data collected during teaching and learning. Learning analytics, for instance, applies known predictive models in instructional systems, using different knowledge, such as information science, sociology and psychology, as well as statistics, AI, and data mining in order to influence educational practice.

Educational data mining

Diving a little bit into the subject, the need for understanding how students learn is the major force behind educational data mining. The suite of computational and psychological methods and research approaches supported by interactive learning methods and tools, such as intelligent tutoring systems, simulations, games, have opened up opportunities to collect and analyze student data and to discover patterns and trends in those data. Data mining algorithms help find variables that can be explored for modelling and by applying data mining methods that classify data and find relationships, these models can be used to change what students experience next or even to recommend outside academic assignments to support their learning.

An important feature of educational data is that they are hierarchical. All the data (from the answers, the sessions, the teachers, the classrooms, etc.) are nested inside one another. Grouping it by time, sequence, and context provide levels of information that can show the impact of the practice sessions length or the time spent to learning – as well as how concepts build on one another and how practice and tutoring should be ordered. Providing the right context to these information help to explain results and to know where the proposed instructional strategy works or not. The methods that have been important to stimulate developments in mining educational data are those related:

1) To prediction, for understanding what behaviors in an online learning environment, such as participation in discussion forums and taking practice tests, can be used to predict outcome such as which students might fail a class. It helps to develop models that provide insights that might help to better connect procedures or facts with the specific sequence and amount of practice items that best stimulate the learning. It also helps to forecast or understand student educational outcomes, such as success on posttests after tutoring.

2) To clustering, meaning to find data points that naturally group together and that can be used to split a full dataset into categories. Examples of clustering are grouping students based on their learning difficulties and interaction patterns, or grouping by similarity of recommending actions and resources.

3) To relationship, meaning discover relationships between variables in a dataset and encoding them as rules for later use. These techniques can be used to associate student activity (in a learning management system or discussion forums) with student grades, to associate content with user types to build recommendations for content that is likely to be interesting or even to make changes to teaching approaches. This latter area, called teaching analytics, is of growing importance and key to discover which pedagogical strategies lead to more effective or robust learning.

4) To distillation, which is a technique that involves depicting data in a way that enables humans to quickly identify or classify features of the data. This area of educational data mining improves machine learning models by allowing humans to identify patterns or features easier, such as student learning actions, student behaviors or collaboration among students.

5) To model discovery, which is a technique that involves using a validated model (developed through such methods as prediction or clustering) as a component in further analysis. Discovery with models supports discovery of relationships between student behaviors and student characteristics or contextual variables, analysis of research questions across a wide variety of contexts, and integration of psychometric modeling into machine learned models.

Learning Analytics

Learning analytics emphasizes measurement and data collection as activities necessary to undertake, understand, analyze and report data with educational purposes. Unlike educational data mining, learning analytics generally does not emphasize reducing learning into components but instead seeks to understand entire systems and to support human decision making. Draws on a broad array of academic disciplines, incorporating concepts from information science, computer science, sociology, statistics, psychology, and learning sciences.

The goal is to answer important questions that affect the way students learn and help us to understand the best way to improve organizational learning systems. Therefore, it emphasizes models that could answer questions such as:

• When are students ready to move on to the next topic?
• When is a student at risk for not completing a course?
• What is the best next course for a given student?
• What kind of help should be provide?

As a visual representation of analytics is critical to generate actionable analyses, the information is often represented as “dashboards” that show data in an easily digestible form. Although the methods used in learning analytics are draw from those used in educational data mining, it may employ additionally social network analysis (to determined student-to-student and student-to-teacher relationships and interactions that help to identify disconnected students, influencers, etc.) and social metadata to determine what a user is engaged with.

As content moves online and mobile devices for interacting with content enable a 24/7 access, understand what data reveal can lead to fundamental shifts in teaching and learning systems as a whole. Learners and educators at all levels can draw benefits from understanding the possibilities of the use of big data in education. Data mining and learning analytics are two powerful tools that can help shape the future of human learning.

References

[1] Anaya, A. R., and J. G. Boticario. 2009. “A Data Mining Approach to Reveal Representative Collaboration Indicators in Open Collaboration Frameworks.” In Educational Data Mining 2009: Proceedings of the 2nd International Conference on Educational Data Mining, edited by T. Barnes, M. Desmarais, C. Romero, and S. Ventura, 210–219.

[2] Amershi, S., and C. Conati. 2009. “Combining Unsupervised and Supervised Classification to Build User Models for Exploratory Learning Environments.” Journal of Educational Data Mining 1 (1): 18–71.

[3] Arnold, K. E. 2010. “Signals: Applying Academic Analytics. EDUCAUSE Quarterly 33 (1). http://www.educause.edu/EDUCAUSE+Quarterly/EDUCAUSEQuarterlyMagazineVolum/SignalsApplyingAcademicAnalyti/199385

[4] Bajzek, D., J. Brooks, W. Jerome, M. Lovett, J. Rinderle, G. Rule, and C. Thille. 2008. “Assessment and Instruction: Two Sides of the Same Coin.” In Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2008, edited by G. Richards. Chesapeake, VA: AACE, 560–565.

[5] Baker, R. S. J. d. 2011. “Data Mining for Education.” In International Encyclopedia of Education, 3rd ed., edited by B. McGaw, P. Peterson, and E. Baker. Oxford, UK: Elsevier.

[6] Baker, R. S. J. d., A.T. Corbett, and V. Aleven. 2008. “More Accurate Student Modeling Through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing.” In Proceedings of the 9th International Conference on Intelligent Tutoring Systems. Berlin, Heidelberg: Springer-Verlag, 406–415.

[7] Baker, R. S. J. d., A.T. Corbett, K. R. Koedinger, and I. Roll. 2006. “Generalizing Detection of Gaming the System Across a Tutoring Curriculum.” In Proceedings of the 8th International Conference on Intelligent Tutoring Systems. Berlin, Heidelberg: Springer-Verlag, 402–411.

[8] Baker, R. S., A. T. Corbett, K. R. Koedinger, and A. Z. Wagner. 2004. “Off-Task Behavior in the Cognitive Tutor Classroom: When Students ‘Game the System.’” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’04). New York, NY: Association for Computing Machinery, 383–390.

[9] Baker, R. S. J. d., S. M. Gowda, and A. T. Corbett. 2011. “Automatically Detecting a Student’s Preparation for Future Learning: Help Use Is Key.” In Proceedings of the 4th International Conference on Educational Data Mining, edited by M. Pechenizkiy, T. Calders, C. Conati, S. Ventura, C. Romero, and J. Stamper, 179–188.

[10] Baker, R. S. J. D., and K. Yacef. 2009. “The State of Educational Data Mining in 2009: A Review and Future Visions.” Journal of Educational Data Mining 1 (1): 3–17.

[11] Balduzzi, M., C. Platzer, T. Holz, E. Kirda, D. Balzarotti, and C. Kruegel. 2010. Abusing Social Networks for Automated User Profiling. Research Report RR-10-233 – EURECOM, Sophia Antipolis; Secure Systems Lab, TU Wien and UCSB.

[12] Beck, J. E., and J. Mostow. 2008. “How Who Should Practice: Using Learning Decomposition to Evaluate the Efficacy of Different Types of Practice for Different Types of Students.” In Proceedings of the 9th International Conference on Intelligent Tutoring Systems.

[13] Bienkowski, Marie; Feng, Mingyu; Means, Barbara. Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics: An Issue Brief. Center for Technology in Learning. SRI International. 2012.

[14] Blikstein, P. 2011. “Using Learning Analytics to Assess Students’ Behavior in Open-Ended Programming Tasks.” Proceedings of the First International Conference on Learning Analytics and Knowledge. New York, NY: Association for Computing Machinery, 110–116.

[15] Brown, W., M. Lovett, D. Bajzek, and J. Burnette. 2006. “Improving the Feedback Cycle to Improve Learning in Introductory Biology Using the Digital Dashboard.” In Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2006I, edited by G. Richards. Chesapeake, VA: AACE, 1030–1035.

[16] Corbett, A. T., and J. R. Anderson. 1994. “Knowledge Tracing: Modeling the Acquisition of Procedural Knowledge.” User Modeling and User-Adapted Interaction 4 (4): 253–278.

[17] Crawford, V., M. Schlager, W. R. Penuel, and Y. Toyama. 2008. “Supporting the Art of Teaching in a Data-Rich, High-Performance Learning Environment.” In Data-Driven School Improvement, edited by E. B. Mandinach and M. Honey. New York, NY: Teachers College Press, 109–129.

[18] Dawson, S., L. Heathcote, and G. Poole. 2010. “Harnessing ICT Potential: The Adoption and Analysis of ICT Systems for Enhancing the Student Learning Experience.” International Journal of Educational Management 24 (2): 116–128.

[19] EDUCAUSE. 2010. Next Generation Learning Challenges: Learner Analytics Premises. http://www.educause.edu/Resources/NextGenerationLearningChalleng/215028

[20] Elias, T. 2011. Learning Analytics: Definitions, Processes and Potential. http://learninganalytics.net/LearningAnalyticsDefinitionsProcessesPotential.pdf

[21] Feng, M., N. T. Heffernan, and K. R. Koedinger. 2009. “User Modeling and User-Adapted Interaction: Addressing the Assessment Challenge in an Online System That Tutors as It Assesses.” The Journal of Personalization Research (UMUAI journal) 19 (3): 243–266.

[22] Gerhard, F. 2001. “User Modeling in Human-Computer Interaction.” User Modeling and User-Adapted Interaction 11: 65–86.

[23] Goldstein, P. J. 2005. Academic Analytics: The Use of Management Information and Technology in Higher Education. EDUCAUSE Center for Applied Research. http://net.educause.edu/ir/library/pdf/ECM/ECM0508.pdf

[24] Graf, S., and Kinshuk. In press. “Dynamic Student Modeling of Learning Styles for Advanced Adaptivity in Learning Management Systems.” International Journal of Information Systems and Social Change.

[25] Hamilton, L., R. Halverson, S. Jackson, E. Mandinach, J. Supovitz, and J. Wayman. 2009. Using Student Achievement Data to Support Instructional Decision Making (NCEE 2009-4067). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance.

[26] Jeong, H., and G. Biswas. 2008. “Mining Student Behavior Models in Learning-by-Teaching Environments.” In Proceedings of the 1st International Conference on Educational Data Mining, Montréal, Québec, Canada,127–136.

[27] Johnson, L., A. Levine, R. Smith, and S. Stone. 2010. The 2010 Horizon Report. Austin, TX: The New Media Consortium. http://wp.nmc.org/horizon2010/

[28] Johnson, L., R. Smith, H. Willis, A. Levine, and K. Haywood. 2011. The 2011 Horizon Report. Austin, TX: The New Media Consortium. http://net.educause.edu/ir/library/pdf/HR2011.pdf

[29] Kardan, S., and C. Conati. 2011. A Framework for Capturing Distinguishing User Interaction Behaviours in Novel Interfaces. In Proceedings of the 4th International Conference on Educational Data Mining, edited by M. Pechenizkiy, T. Calders, C. Conati, S. Ventura, C. Romero, and J. Stamper, 159–168.

[30] Köck, M., and A. Paramythis. 2011. “Activity Sequence Modeling and Dynamic Clustering for Personalized E-Learning. Journal of User Modeling and User-Adapted Interaction 21 (1-2): 51–97.

[31] Koedinger, K. R., R. Baker, K. Cunningham, A. Skogsholm, B. Leber, and J. Stamper. 2010. “A Data Repository for the EDM Community: The PSLC DataShop.” In Handbook of Educational Data Mining, edited by C. Romero, S. Ventura, M. Pechenizkiy, and R.S.J.d. Baker. Boca Raton, FL: CRC Press, 43–55.

[32] Koedinger, K., E. McLaughlin, and N. Heffernan. 2010. “A Quasi-experimental Evaluation of an On-line Formative Assessment and Tutoring System.” Journal of Educational Computing Research 4: 489–510.

[33] Lauría, E. J. M., and J. Baron. 2011. Mining Sakai to Measure Student Performance: Opportunities and Challenges in Academic Analytics. http://ecc.marist.edu/conf2011/materials/LauriaECC2011-%20Mining%20Sakai%20to%20Measure%20Student%20Performance%20-%20final.pdf

[34] Long, P. and Siemens, G. 2011. “Penetrating the Fog: Analytics in Learning and Education.” EDUCAUSE Review 46 (5).

[35] Lovett, M., O. Meyer, and C. Thille. 2008. “The Open Learning Initiative: Measuring the Effectiveness of the OLI Statistics Course in Accelerating Student Learning.” Journal of Interactive Media in Education Special Issue: Researching Open Content in Education. 14. http://jime.open.ac.uk/2008/14.

[36] Macfayden, L. P., and S. Dawson. 2010. “Mining LMS Data to Develop an ‘Early Warning’ System for Educators: A Proof of Concept.” Computers & Education 54 (2): 588–599.

[37] Manyika, J., M. Chui, B. Brown, J. Bughin, R. Dobbs, C. Roxburgh, and A. H. Byers. 2011. Big Data: The Next Frontier for Innovation, Competition, and Productivity. McKinsey Global Institute. http://www.mckinsey.com/Insights/MGI/Research/Technology_and_Innovation/Big_data_The_next_frontier_for_innovation

[38] Martin, B., A. Mitrovic, K. Koedinger, and S. Mathan. 2011. “Evaluating and Improving Adaptive Educational Systems with Learning Curves.” User Modeling and User-Adapted Interaction 21 (3): 249–283.

[39] Means, B., C. Chelemer, and M. S. Knapp (eds.). 1991. Teaching Advanced Skills to at-Risk Students: Views from Research and Practice. San Francisco, CA: Jossey-Bass.

[40] Merceron, A., and K. Yacef. 2010. “Measuring Correlation of Strong Symmetric Association Rules in Educational Data.” In Handbook of Educational Data Mining, edited by C. Romero, S. Ventura, M. Pechenizkiy, and R. S. J. d. Baker. Boca Raton, FL: CRC Press, 245–256.

[41] New Media Consortium. 2012. NMC Horizon Project Higher Ed Short List. Austin, TX: New Media Consortium. http://www.nmc.org/news/download-horizon-project-2012-higher-ed-short-list.

[42] O’Neil, H. F. 2005. What Works in Distance Learning: Guidelines. Greenwich CT: Information Age Publishing.

[43] Reese, D. D., R. J. Seward, B. G. Tabachnick, B. Hitt, A. Harrison, and L. McFarland. In press. “Timed Report Measures Learning: Game-Based Embedded Assessment.” In Assessment in Game-Based Learning: Foundations, Innovations, and Perspectives, edited by D. Ifenthaler, D. Eseryel, and X. Ge. New York, NY: Springer.

[44] Ritter, S., J. Anderson, K. Koedinger, and A. Corbett. 2007. “Cognitive Tutor: Applied Research in Mathematics Education.” Psychonomic Bulletin & Review 14 (2): 249–255.

[45] Romero C. R., and S. Ventura. 2010. “Educational Data Mining: A Review of the State of the Art.” IEEE Transactions on Systems, Man and Cybernetics, Part C: Applications and Reviews 40 (6): 601–618.

[46] Siemens, G., and R. S. J. d. Baker. 2012. “Learning Analytics and Educational Data Mining: Towards Communication and Collaboration.” In Proceedings of LAK12: 2nd International Conference on Learning Analytics & Knowledge, New York, NY: Association for Computing Machinery, 252–254.

[47] U.S. Department of Education. 2010a. National Education Technology Plan. http://www.ed.gov/technology/netp-2010.

———. 2010b. Use of Education Data at the Local Level: From Accountability to Instructional Improvement. Washington, DC: U.S. Department of Education.

———. 2010c. Basic Concepts and Definitions for Privacy and Confidentiality in Student Education Records. SLDS Technical Brief 1. NCES 2011-601. Washington, DC: U.S. Department of Education.

———. 2012a. December 2011- Revised FERPA Regulations: An Overview for SEAS and LEAS. (PDF file). Washington, DC: U.S. Department of Education. http://www.ed.gov/policy/gen/guid/fpco/pdf/sealea_overview.pdf

———. 2012b. The Family Educational Rights and Privacy Act: Guidance for Reasonable Methods and Written Agreements (PDF file). Washington, DC: U.S. Department of Education. http://www.ed.gov/policy/gen/guid/fpco/pdf/reasonablemtd_agreement.pdf

[48] U.S. Department of Education, Office of Educational Technology, Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics: An Issue Brief, Washington, D.C., 2012.

[49] VanLehn, K., C. Lynch, K. Schulze, J. A. Shapiro, R. H. Shelby, L. Taylor, D. Treacy, A. Weinstein, and M. Wintersgill. 2005. “The Andes Physics Tutoring System: Lessons Learned.” International Journal of Artificial Intelligence in Education 15 (3): 147–204.

[50] Viégas, F. B., M. Wattenberg, M. McKeon, F. Van Ham, and J. Kriss. 2008. “Harry Potter and the Meat-Filled Freezer: A Case Study of Spontaneous Usage of Visualization Tools.” In Proceedings of the 41st Annual Hawaii International Conference on System Sciences, 159.

[51] Wayman, J. C. 2005. “Involving Teachers in Data-Driven Decision Making: Using Computer Data Systems to Support Teacher Inquiry and Reflection.” Journal of Education for Students Placed At Risk 10 (3): 295–308.