Data-Unification is a project that helps users to import data from multiple sources such as Excel, MS SQL Server, and MySQL databases into a single target database. The project uses SQL Server Integration Services (SSIS) as the primary tool for data integration, but it also provides an alternate method using Apache NiFi. The project aims to simplify the process of unifying financial data from multiple sources, which is a common challenge faced by financial analysts and data scientists.
To install Financial-Data-Unification, you must have SQL Server Integration Services and Apache NiFi installed on your machine. You can download SQL Server Integration Services from the Microsoft website, and Apache NiFi can be downloaded from the Apache NiFi website. Once you have installed these tools, you can clone the repository from GitHub and open the project in Visual Studio or any other IDE that supports SSIS.
The Financial-Data-Unification project can be used to import financial data from Excel, MS SQL Server, and MySQL databases into a single target database. To use the project, follow these steps:
- Open the SSIS package or NiFi flow in Visual Studio or NiFi respectively.
- Configure the connection managers for the source and target databases.
- Modify the package or flow to customize the data import process if necessary.
- Execute the package or flow to import data into the target database.
The Financial-Data-Unification project supports the following data sources:
- Excel spreadsheets (XLS, XLSX)
- MS SQL Server databases
- MySQL databases The project requires the appropriate ODBC or JDBC drivers to connect to these data sources. The data sources must also be accessible from the machine where the project is installed.
The Financial-Data-Unification project is primarily implemented using SQL Server Integration Services (SSIS) and Apache NiFi. The SSIS package uses data flow components such as the OLE DB Source, Excel Source, and ADO.NET Destination to import data from the source databases and load it into the target database. The Apache NiFi flow uses processors such as GetSQL, QueryDatabaseTable, and PutSQL to achieve the same functionality.
Contributions to the Data-Unification project are welcome. To contribute, follow these steps:
- Fork the repository on GitHub.
- Create a new branch for your changes.
- Make changes to the code or documentation.
- Submit a pull request with a detailed description of your changes.
The Financial-Data-Unification project is maintained by Abhilash Panicker. It uses the following external libraries:
- Apache NiFi
- NPOI
- Newtonsoft.Json
The Financial-Data-Unification project is licensed under the MIT license. See the LICENSE file for details.
To report bugs or provide feedback, please contact Abhilash Panicker at here
