Here’s a simple script to combine all csv files in a folder into a single file. concat_data_files.py (2.6 KB)
I’m going to leave pretty explicit instructions for future readers.
Create a folder to work in
To follow this example here, create a folder on your Desktop called concat_files. Inside that, create a folder called ‘input_files’ and another called ‘output_files’. Copy the python script included here into the ‘concat_files’ folder. Copy all of the files you want to combine into the ‘input_files’ folder.
You need to have pandas installed. To check for pandas, open a terminal, enter this:
>>> import pandas
If you don’t get an error, you have pandas installed.
If you don’t have pandas installed, I highly recommend you use a virtual environment. Virtualenv is the most common, but since we’re all in the sciences here I’d recommend using Anaconda. If you don’t have Anaconda installed, you can start with Miniconda, it’s a smaller download.
Create a virtual environment with conda (optional)
If you’re using conda, create a new folder somewhere, copy this file into it. Open a terminal and
cd into it. For example, if you created a folder ‘concat_files’ on your Desktop, type this in Mac / Linux:
Create a conda virtual environment.
conda create -n "concat_files" python=3.6
Activate the conda environment
conda activate concat_files
conda install pandas
Or using pip:
pip install pandas
Run the script
If you’re following along, just type this:
You can always edit the script to your liking.