distfeatselect: distributed feature selection
Dzeneta Kudumovic, Aida Brankovic
distfeatselect is a Python package implementing an innovative Distributed Feature Selection algorithm based on vertical data partitioning and distributed searching. By dividing features into subsets and assigning dedicated processors for local searches, the algorithm achieves parallelism and scalability, making it suitable for large-scale datasets. The distributed architecture enhances efficiency by operating on smaller search spaces, reducing computational time. Moreover, algorithm's tendency to produce simple model structures enhances interpretability and robustness.
The DFS (Distributed Feature Selection) algorithm is implemented as a scikit-learn-compatible transformer, adhering to the scikit-learn API standards. This enables seamless integration into scikit-learn workflows, empowering users to incorporate distributed feature selection into their machine learning pipelines.
To get started with distfeatselect, explore our provided notebooks, which offer hands-on examples and demonstrations of the package's functionality.
The package dcor depends on the following libraries:
distfeatselect is on PyPi and can be installed using pip:
pip install distfeatselectThe algorithms integrated into these packages have their foundations in rigorous academic research. Specifically, the methodology employed is derived from the research papers [1] and [2]. By implementing the insights and techniques outlined in this scholarly work, the packages aim to provide users with efficient and effective solutions informed by the latest advancements in the field.
Additionally, users interested in further understanding the underlying methodologies are encouraged to refer to the following research papers:
This project is licensed under the MIT License.