Result: Comparison of Multiprocessor and Multi-Threaded Implementations of the Entropy Approach to Impute Gaps in Data in Python

Title:
Comparison of Multiprocessor and Multi-Threaded Implementations of the Entropy Approach to Impute Gaps in Data in Python
Source:
Challenges and Issues of Modern Science; Vol. 2 (2024): Challenges and Issues of Modern Science; 300-304; 3083-5704; 10.15421/cims.2
Publisher Information:
Oles Honchar Dnipro National University 2024-06-14
Document Type:
Electronic Resource Electronic Resource
Availability:
Open access content. Open access content
https://creativecommons.org/licenses/by/4.0
Note:
application/pdf
Ukrainian
Other Numbers:
UAOHD oai:ojs2.cims.fti.dp.ua:article/131
1476141422
Contributing Source:
OLES HONCHAR DNIPRO NAT UNIV
From OAIsterĀ®, provided by the OCLC Cooperative.
Accession Number:
edsoai.on1476141422
Database:
OAIster

Further Information

The comparison of sequential, multi-processor, and multi-threaded implementations of the entropy-based approach for data imputation is being considered in Python programming language. The main goal of the work is to investigate approaches to optimizing computations when implementing the entropy-based approach for data imputation. The authors explain the limitations of Python's interpreter regarding multi-threading due to the presence of the Global Interpreter Lock (GIL), which prevents full parallel data processing in a multi-threaded environment. Instead, they propose using multi-processor calculating, where each process has its own Python interpreter and GIL, allowing for efficient distribution of computational tasks across multiple processor cores. For the experimental part of the work, the UCI Heart Disease Data dataset, hosted on the Kaggle platform, is used. Artificial introduction of gaps is performed, then imputation using various implementations based on the entropy approach, assessing the accuracy of imputation and the runtime of algorithms. The authors consider three approaches: sequential, multi-threaded, and multi-processor, and compare their efficiency. The research results show that the multi-threaded approach does not provide an advantage in speed compared to the sequential approach and sometimes even worsens productivity due to time spent on thread switching. Conversely, the multi-processor approach demonstrates a reduction in computation time, confirming its effectiveness for data imputation tasks. In the conclusions, the authors note that optimizing computations in Python requires consideration of GIL peculiarities and recommend using multi-processor computations to achieve better productivity. Recommendations for further optimization are provided, including the use of vectorized computations and avoiding excessive input-output operations. This work is of practical importance for data science researchers working with Python and facing challenges i