Use of Bayesian Networks in Brazil High School Educational Database: Analysis of the Impact of Covid-19 on ENEM in Para between 2019 and 2022

Sandio Maciel Dos Santos^*
*Correspondence: Sandio Maciel Dos Santos, Department of Science, Federal University of Para, Belem, Para, Brazil, Email:

Keywords

COVID-19 • Educational data mining • ENEM • Education

Introduction

In the attempt to manage confirmed cases of COVID-19, it became evident that there was no linear relationship between infections and social impact across various fundamental sectors of society, including health, education, and urban infrastructure [1]. This disparity in infection rates is closely linked to the levels of social inequality prevalent in affected populations, directly contributing to virus transmission [2-4]. Consequently, numerous efforts have been made to alleviate the repercussions of the COVID-19 pandemic, with education being one of the most profoundly affected sectors, leading to the adoption of remote learning initiatives [5-7].

Furthermore, studies such as Ariyo E, et al. [8] conducted a descriptive-analytical study utilizing binomial regression in Nigeria to elucidate the social dynamics between the surge in COVID-19 cases and remote learning performance among elementary education students. The study incorporated various social dimensions, including education level, gender, ethnicity, family composition, etc., across Nigeria's six most influential geopolitical zones. Findings from this research Ariyo E, et al. [8] revealed that family composition significantly influences the rise in COVID-19 infections, along with family income, where higher incomes correlate with better student performance.

Similarly, research such as Zhu W, et al. [9] employed electronic surveys with responses from parents of Chinese children, representing geographically diverse regions: Oriental (27.3%), Central (38.8%), and Western (33.9%) China. The study collected social data such as age, relationship type, education level, family income, etc., to profile the students. Descriptive statistical analysis revealed that 51.3% of children experienced fatigue due to the extensive subjects and prolonged screen time, leading to discomfort and stress. Moreover, the majority (85.4%) reported that remote education failed to match the level of learning provided by face-to-face teaching.

In Brazil, various programs evaluate students' educational progress across both public and private networks, spanning from basic to higher education. The National Institute for Educational Studies and Research Anísio Teixeira (INEP) administers annual evaluation programs, serving as indicators of education quality throughout students' educational journey [10]. These evaluation programs include the Educational Census, ENADE, ENEM, etc., tailored to each educational cycle to gauge students' learning levels [11].

Given the socio-educational implications of the COVID-19 pandemic, it is evident that students completing their final years of high school during the pandemic experienced disparate performances, with a notable increase in ENEM dropouts, possibly linked to individual social inequalities [12].

Thus, this study proposes a data analysis using Data Science techniques to mine educational data from the microdata of ENEM from 2019 to 2022, examining pre-pandemic and pandemic-era COVID-19 periods, to ascertain the social dimensions exerting the greatest influence on student performance, particularly in the state of Pará.

The document is structured as follows: Section 2 outlines the materials and methods employed in the study; Section 3 details the results obtained using Bayesian networks. Finally, Section 4 provides a discussion on the findings.

Methodology

The methodology employed in this study involves the application of data science techniques, specifically Educational Data Mining [13,14] as the primary means of extracting knowledge from databases, to utilize the acquired information for decision-making purposes. The analysis focuses on educational data about high school students and graduates to investigate the impacts of the COVID-19 pandemic.

For this study, databases from the ENEM exams of 2019 (pre-pandemic) and 2020-2022 (pandemic period) are selected. These years were chosen due to the significant increase in COVID-19 infections, along with the respective school census K_{Q ≤ 25%}es for the same periods, serving as microdata sources for the ENEM. This selection enables an examination of student performance amidst the challenges posed by the pandemic, particularly in the context of national exam resolutions aimed at determining the influence of school closures during periods of heightened epidemic risk [15-17].

The study categorizes performance using quartiles based on the minimum and maximum scores within each area of knowledge [18,19] as well as the number of dropouts per edition of the ξ exam. In this context, represents a vector of numbers ordered in ascending order, Q indicates the analyzed percentile, and P ranges from 1 to 3, determining the percentiles of each quartile K (Equation 1).

Unlike previous studies that have solely relied on average scores as a performance criterion [20-23] this study adopts a more comprehensive approach. It categorizes students' performance in the ENEM exam into four distinct groups, as illustrated in Table 1.

Table 1 illustrates the discretization into three groups using the quartile method: K_{Q ≤ 25%} for scores below 25%, 25% < K_Q<75% for scores between 26% and 74%, and K_Q>75% for scores above 75%. The variable ξ represents the number of dropouts. This demonstration is crucial for understanding the realistic impacts of COVID-19 on sociodemographic dimensions and their influence on student performance during educational interruptions.

The microdata from ENEM for 2019 and 2022 comprise 2.24, 1.88, and 1.40 gigabytes, respectively, each containing a set of 76 variables. In total, they represent more than 14 million instances, reflecting the number of exam participants nationwide. Among the 80 indicators analyzed, 22 were selected from the dataset as they showed a higher correlation with score performance. This aimed to streamline the construction of the representative Bayesian Network (BN) [24] for the problem at hand.

The variables representing scores in different areas of knowledge are grouped into four performance analysis groups, as depicted in Table 1. For the variable Monthly Family Income (Q006), which consists of income brackets (e.g., "from R$ 0.00 to R$ 998.00"), we opted to use the lowest salary and the number of people per household (Q005) to replace the original text and group them accordingly, as per the ENEM variables dictionary [10].

The data mining phase employs the Bayesian Network technique through the PGMPY library [25] due to its ease of configuration and usability, as well as its intuitive generation of probabilistic relationships and display of Conditional Probability Tables (CPTs) for each node. For visualization, the pyAgrum API is utilized [26]. Ultimately, statistical and probabilistic inferences drawn from the microdata from ENEM and the School Census aim to compare the sociodemographic effects of successive epidemic outbreaks, confirmed cases, and deaths on student performance. This includes identifying those most likely to be affected when a public health alert is declared.

Results

In this case study, we examine the repercussions of the COVID-19 pandemic on the academic performance of high school students, specifically focusing on those in the state of Pará, in the context of the National High School Examination (ENEM) in Brazil. The analysis utilizes microdata from the ENEM editions of 2019 and 2022, alongside microdata from the corresponding School Census in each respective year.

The representative microdata for the state of Pará encompass approximately 279.593, 330.883, and 185.978 participants for each analyzed edition, with a negligible difference of 0.05% from the official numbers. As outlined in Table 2, the analysis comprises 12 pertinent parameters selected during the data processing phase. These variables shed light on the principal socio-behavioral changes among participants, influencing overall scores during the ENEM amid the COVID-19 pandemic in Brazil (Table 2).

As depicted in Table 2, participants whose family income does not exceed approximately 1 minimum wage exhibit the highest dropout rates from the ENEM during the years 2020 and 2021 compared to the previous year. This underscores the disproportionate impact of non-seasonal respiratory epidemic outbreaks, such as the COVID-19 pandemic, on individuals with lower income, given the prolonged closures of public institutions [27-29].

Furthermore, in a separate analysis involving participants with an overall score above 75%, those attending or previously enrolled in private schools during the pandemic demonstrated superior performance compared to their counterparts from public schools. Additionally, higher maternal occupation and education levels were associated with better student performance. These findings suggest a correlation between the availability of resources, such as computers and internet access, and student performance, as illustrated in Table 3.

The analysis of data from students scoring above 75% by administrative dependence reveals disparities in performance before the pandemic. Participants from public schools, especially those with mothers having elementary education and lower occupational status, exhibited lower performance compared to their counterparts from private institutions, whose mothers had higher levels of education and engaged in occupations requiring advanced education [30,31].

This initial analytical process aimed to elucidate the influence of social parameters on the performance of participants from Pará in the ENEM. To explore how the increase in cases of respiratory syndrome during the COVID-19 pandemic affected student performance, a Bayesian probabilistic analysis was conducted [32]. This involved employing techniques such as Hill-ClimbSearch, K2Score, and VariableElimination [33] with support from the pyAgrum library to visualize the Bayesian Networks (BNs) for the years 2019 and 2022, as depicted in Figure 1.

The BN derived from the 2019 data highlights key variables significantly impacting the performance of ENEM participants, including the father's education level, family income, computer dependency, and administrative dependence in the household, as depicted in Figure 1a. This organizational structure establishes a flow of probabilistic dependencies among the selected parameters. Applying the same methodology to structure BNs for the educational data from 2020 and 2022 (Figures 1b and 1c), directly influenced by the COVID-19 pandemic, reveals a notable shift: the presence of a computer in the household is no longer a primary variable of importance.

The selection of these parameters is determined by the BN itself, which identifies the most critical conditional dependencies for analysis. Consequently, when examining the inferences drawn from these variables, it becomes evident that higher levels of father's education correlate with better performance of participants (*) (Table 4) and [34]. On the other hand, the number of participants who abandoned the exam (ξ) increased by 19%, between the pre-post COVID-19 period, for parents with primary education and 8%, between the pre-COVID-19 period -post COVID-19, for those with higher education, and during the COVID-19 pandemic in 2020, dropout rates are approximately 31% for primary education and 10% for higher education, in 2020 and in 2021 they are approximated at 14% and 9% respectively (Table 4).

An important aspect to highlight is the conditional probability between administrative dependence and the availability of a computer in the household for educational purposes. Inferences reveal a notable correlation, particularly among public school students who have access to a computer, showing a significant relationship with their performance on ENEM scores. When assessing the scores of students classified in Group K_Q<75%, a noteworthy disparity is observed between those with and without access to computers, indicating a significant increase in performance for the former. Specifically, among private school students, there is a 13% increase, as illustrated in Table 5.

Another crucial aspect to consider is the conditional probability between administrative dependence and the availability of a household computer for educational activities. Inferences indicate a significant correlation, particularly among public school students who have access to computers, showing notable improvements in ENEM scores compared to those without access. For students classified in Group K_Q<75%, there is a noteworthy increase in scores among those with access to computers. Specifically, private school students exhibit a 20% increase, as detailed in Table 5.

Furthermore, an analysis of participants' declared family income reveals a strong relationship between higher family income (C6) (*) and student scores, as illustrated in Table 6. Consistent with this inference, an examination of pre-established family income ranges demonstrates a decline in performance among students reporting incomes of up to 1 salary (C1). But those achieving scores of K_Q≥75% notably, there was a reduction of approximately 6.5% among the number of participants in this income range.

In a more detailed analysis, the impact of performance is assessed concerning the administrative dependence and family income of the participants. It becomes evident that the proportion of students in public schools decreased to Group K_Q≥75% when linked to incomes of up to 1 minimum wage. Conversely, the number of dropouts increased by 30%. Figure 2 provides a visual representation of the performance of participants based on family income (Figure 2).

In Figure 2, a notable increase in participants in the K_Q<75% Group between 2020 and 2021 from private schools is observed. This migration may be attributed to the challenges posed by remote learning during the peaks of confirmed COVID-19 cases in Brazil. In contrast, students from public schools account for the majority of dropouts in the national exam [31].

A more specific analysis of educational data from the state of Pará, focusing on the relationship between its six mesoregions and the school census, sheds light on whether the impact of the COVID-19 pandemic had uniform effects on abstention rates and overall participant performance. Figure 3 displays the mesoregions of Pará for clearer reader comprehension, alongside the percentages of abstention among ENEM participants in the 2019 and 2022 editions (Figure 3).

Among the mesoregions of Pará depicted in Figure 4, it is noteworthy that only the Belém Metropolitan Region and the Northeastern Pará region experienced a significant rebound in reducing abstention rates in the ENEM during the COVID-19 pandemic between 2020 and 2021. Conversely, the other regions maintained persistently high dropout rates from the ENEM during the same period, with percentages exceeding 20% (Figure 4).

The Marajó region emerged as one of the most severely affected areas following the onset of the COVID-19 pandemic. Notably, between 2020 and 2022, students from public schools experienced a substantial decline in performance, with fewer than 10% achieving scores above 75% in the assessments. Moreover, it is crucial to highlight the surge in absent students during the ENEM exams from private schools. This trend could be linked to travel restrictions resulting from lockdowns and the closure of educational institutions on the island, as illustrated in Figure 4.

Discussion

The case study presented in this article addresses the systematic relationship between waves of COVID-19 infection and the Brazilian educational system, with a specific focus on the state of Pará. It analyzes the main impacts of interruptions in both public and private high school institutions. The analysis is based on the performance of students in the last year of high school who take the ENEM exam. The analysis of ENEM educational data revealed that students with lower per capita income (up to 1 minimum wage) faced greater difficulties in acquiring the curriculum content. This trend is evidenced by Table 6, which shows that the lower the income, the higher the percentage of dropout students. Additionally, it is important to consider other correlated factors, such as population density per square meter, where a higher number of people per area may increase the chances of virus transmission. Another relevant aspect is the relationship between the administrative dependence of schools and the level of education of parents, which also influences students' performance on the ENEM. It was noticed that students from public schools showed a higher dropout rate when their parents had lower levels of education than high school [35,36].

Conclusion

It is noteworthy that private educational institutions had a smaller impact on the teaching-learning process of their students compared to public institutions. Furthermore, despite small positive or negative fluctuations, the differences in the percentage of students who obtained scores above 75% are very small. A relevant data point is related to the performance of private institutions in Pará on the ENEM. In 2019, these institutions occupied the top positions in the ENEM ranking and maintained that position in 2020 and 2022, even with the transition to remote learning. This highlights the ability of private institutions to maintain high performance standards for their students, even in the face of challenges such as distance learning. Thus, variables such as population density and the educational level of parents also significantly influenced student performance. While private educational institutions showed less disruption in the teaching-learning process compared to public ones, consistently maintaining high performance standards in the ENEM, even during periods of remote learning.

Supplementary Information

None declared.

Acknowledgement

Thanks to Hydro for the support and funding of this survey. Since 2019, the company has collaborated with UFPA in several initiatives through a technical and scientific cooperation agreement.

To CNPq - National Council for Scientific and Technological Development and CAPES (Coordination for the Improvement of Higher Education Personnel), for funding my research through a scholarship.

Conflict of Interest

The authors declare that they have no competing interests.

References

1. Green, Heidi, Ritin Fernandez and Catherine MacPhail. "The social determinants of health and health outcomes among adults during the COVID‐19 pandemic: A systematic review." Public Health Nurs 38 (2021): 942-952.

Google Scholar, Crossref, Indexed at

2. Paremoer, Lauren, Sulakshana Nandi, Hani Serag and Fran Baum. “Covid-19 pandemic and the social determinants of health.” BMJ 372 (2021):129.

Crossref, Indexed at

3. Park, Ashley and Omer A. Awan. "COVID-19 and virtual medical student education." Acad Radiol 30 (2023): 773-775.

Google Scholar, Crossref, Indexed at

4. Julio, Silva and Marcelo Ribeiro-Alves. “Social inequalities and the pandemic of COVID-19: The case of Rio de Janeiro.” J Epidemiol Community Health 75 (2021):975–979.

Crossref, Indexed at

5. Alqahtani, Ammar Y., and Albraa A. Rajkhan. "E-learning critical success factors during the covid-19 pandemic: A comprehensive analysis of e-learning managerial perspectives." Educ Sci 10 (2020):216.

Google Scholar, Crossref, Indexed at

6. Godoy, Lívea Dornela, Raquel Falcoski, Roberta Monteiro Incrocci and Fabiana Maris Versuti, et al. "The psychological impact of the COVID-19 pandemic in remote learning in higher education." Educ Sci 11 (2021): 473.

Google Scholar, Crossref, Indexed at

7. Cássia, do Carmo Andrade Lisbôa and Maria Cristina do Amaral Moreira “A avaliação da aprendizagem durante o ensino remoto imposto pela pandemia da Covid-19.” Revista Contexto & Educacao 39 (2024): e14253-e14253.

Crossref, Indexed at

8. Ariyo, Esther, Micheal Amurtiya, Olaleye Yemisi Lydia and Ariyo Oludare, et al. "Socio-demographic determinants of children home learning experiences during COVID 19 school closure." Int J Educ Res Open 3 (2022): 100111.

Google Scholar, Crossref, Indexed at

9. Zhu, Wenting, Qianqian Liu and Xiumin Hong. "Implementation and challenges of online education during the COVID-19 outbreak: A national survey of children and parents in China." Early Child Res Q 61 (2022): 209-219.

Google Scholar, Crossref, Indexed at

10. Brasil. “National institute of educational studies and research anísio teixeira — national institute of educational studies and research anísio teixeira - INEP.” (2024).
11. Neto, Nelson Weber, Raimundo C. Soares, Luciano Reis Coutinho and Ariel Soares Teles. "A Pandemia da COVID-19 impactou o ENEM? Uma Analise Comparativa de Dados dos Anos de 2019 e 2020." Revista Novas Tecnol Educ 20 (2022): 223-232.

Google Scholar, Crossref, Indexed at

12. Dayene, Mendes Silva Gonçalves and Luiz Ismael Pereira. “Abandono escolar no ensino médio: uma análise comparativa antes e durante a pandemia em minas gerais.” Políticas Educacionais 18 (2024):1-22.

Crossref, Indexed at

13. Andrea, Filatro. “Data Science na Educacao: Presencial, a Distancia e Corporativa: Presencial, a Distância e Corporativa.”Saraiva Educacao (2020).
14. Mouromtsev, Dmitry and Mathieu d'Aquin. “Open data for education: Linked, shared, and reusable data for teaching and learning.” 1st ed. 2016 edition ed. Switzerland, Springer (2016).

Crossref

15. Laertes, Pereira Junior, Simone Nasser Matos and Helyane Bronoski Borges. "Socio-economic status and COVID-19–related cases and fatalities." Pub Health 189 (2020):129-134.

Google Scholar, Crossref, Indexed at

16. Karakose, Turgut. "The impact of the COVID-19 epidemic on higher education: Opportunities and implications for policy and practice." Educ Process Int J 10 (2021):7-12.

Google Scholar, Crossref, Indexed at

17. Reimers, Fernando M. "Learning from a pandemic. The impact of COVID-19 on education around the world." Primary and secondary education during Covid-19: Disruptions to educational opportunity during a pandemic (2022): 1-37.

Google Scholar, Crossref, Indexed at

18. Bendikson, Linda and John Hattie and Viviane Robinson. "Identifying the comparative academic performance of secondary schools." J Educ Admin 49 (2011):433-449.

Google Scholar, Crossref, Indexed at

19. Waheed, Hajra, Saeed-Ul Hassan, Naif Radi Aljohani and Julie Hardman, et al. "Predicting academic performance of students from VLE big data using deep learning models."Comput Hum Behav 104 (2020):106189.

Google Scholar, Crossref, Indexed at

20. Boneti, Lindomar Wessler and Geisa Melo Oliveira. "Enem: Análise do desempenho escolar nas edições de 2009 a 2013." Revista Espaço Pedagógico 24 (2017).

Crossref, Indexed at

21. Guilherme, Ferrari Bravin, Luciana Lee and Silvia das Dores Rissino. “Mineraçao de dados educacionais na base de dados do enem 2015.” Brazilian J Prod Eng 5 (2019):186–201.
22. Rafael, Vinicios do Carmo, Wesllei Felipe Heckler and Juliano Varella de Carvalho. “Uma Análise do Desempenho dos Estudantes do Rio Grande do Sul no ENEM 2019.” Revista Novas Tecnol Educ 18 (2020): 378-387.

Crossref, Indexed at

23. Fernando, Lang da Silveira, Marcia Cristina Bernardes Barbosa, Roberto da Silva. “Exame Nacional do Ensino Médio (ENEM): Uma análise crítica.” Revista Brasileira de Ensino de Física 37 (2015):1101.
24. Murphy, Kevin Patrick. “Dynamic Bayesian networks: Representation, inference and learning.” University of California, Berkeley (2002).

Google Scholar, Indexed at

25. Ankan, Ankur and Abinash Panda. “Pgmpy: Probabilistic graphical models using python.” SciPy (2015):6-11.

Google Scholar

26. Ducamp, Gaspard, Christophe Gonzales and Pierre-Henri Wuillemin. "aGrUM/pyAgrum: A toolbox to build models and algorithms for probabilistic graphical models in python." Inter Conf Probabilist Graph Models. PMLR (2020).

Google Scholar

27. Janderson Ferreira Dutra, Joao Batista Firmino Júnior and Damires Yluska de Souza Fernandes. “Fatores que podem interferir no desempenho de estudantes no ENEM: uma revisão sistemática da literatura.” Revista Brasileira de Informatica Educacao 31 (2023): 323-351.

Crossref, Indexed at

28. Cesar, Augusto Andrade Ferreira, Tarcísio da Costa Lobato and Brena do Nascimento Carvalho. “Enem no Norte do Brasil: Uma análise comparativa dos determinantes do desempenho entre o ano de 2019 e o ano pandêmico de 2020.” (2022).
29. Torres, Ronaldo, Mateus Machado de Pereira, Reisoli Bender Filho and Fernanda Cigainski Lisbinski. “Determinantes do desempenho dos participantes da prova do enem: Evidências para o rio grande do sul.” Desenvolvimento em Questao 18 (2020): 352–368.

Google Scholar, Crossref

30. Luanda, Carol de Sousa Fernandes, Francisco Bezerra Mendes, Jodean Alves da Silva and Rodrigo Fonseca Da Silva. “Análise do desempenho em matemática e suas tecnologias dos participantes do ENEM 2021 em Barra do Corda, Maranhão: Uma comparação entre alunos de escolas públicas e privadas por meio de regressão logística.” Contribuciones Las Ciencias Sociales 16 (2023): 33822-33835.

Crossref, Indexed at

31. Diego, Navarro, Matheus Ianello, Felipe Muneratto and Graciella Watanabe. “Impactos do Conhecimento das Ciências Naturais para o Desempenho no ENEM: Considerações sobre a Desigualdade Científico-tecnológica para a Justica Social.” Revista Brasileira de Pesquisa em Educacao em Ciencias (2021): e26002-30.

Crossref, Indexed at

32. Mark, Chavira and Adnan Darwiche. “Compiling Bayesian networks using variable elimination.” IJCAI  2443 (2007).

Google Scholar, Indexed at

33. Koller, Daphne and Nir Friedman. “Probabilistic graphical models: Principles and techniques.” MIT Press (2009).

Google Scholar

34. Suhonen, Tuomo and Hannu Karhunen. "The intergenerational effects of parental higher education: Evidence from changes in university accessibility." J Pub Econ 176 (2019): 195-217.

Google Scholar, Crossref, Indexed at

35. Leandro, Morais Coelho de Oliveira, Luciane Zanin and Flavia Florio. “Professores do ensino fundamental público: Literacia em saúde e fatores associados.” Revista Contexto & Educacao 39 (2024):e13673–e13673.

Crossref, Indexed at

36. Laertes, Pereira Junior, Simone Nasser Matos and Helyane Bronoski Borges. “Análise dos Perfis de Alunos do Ensino Superior sobre a Realização de Aulas na Modalidade a Distância Durante Pandemia da Covid-19 Usando Algoritmos de Aprendizagem de Máquina.” Revista Novas Tecnologias na Educação 18 (2020):336–345.

Indexed at