jsonmergeutils

Merges a series of files containing JSON array of Objects into a single file containing one JSON object.

View on GitHub

jsonmergeutils

CodeQL Pylint

Merges a series of files containing JSON array of Objects into a single file containing one JSON object.

Consider a file with name tes1.json with contents (Refer next section to know more about the arguments accepted):

{
	"states": [{
		"name": "Alabama",
		"abbreviation": "AL",
		"area_codes": ["205", "251", "256", "334", "938"]
	}, {
		"name": "Alaska",
		"abbreviation": "AK",
		"area_codes": ["907"]
	}]
}

and other file with same prefix but different number as suffix say tes2.json with contents:

{
	"states": [{
		"name": "Arizona",
		"abbreviation": "AZ",
		"area_codes": ["480", "520", "602", "623", "928"]
	}, {
		"name": "Arkansas",
		"abbreviation": "AR",
		"area_codes": ["479", "501", "870"]
	}]
}

the resultant file generated by merging tes1.json and tes2.json is:

{"states": [{"name": "Alabama", "abbreviation": "AL", "area_codes": ["205", "251", "256", "334", "938"]}, {"name": "Arkansas", "abbreviation": "AR", "area_codes": ["479", "501", "870"]}, {"name": "Alaska", "abbreviation": "AK", "area_codes": ["907"]}, {"name": "Arizona", "abbreviation": "AZ", "area_codes": ["480", "520", "602", "623", "928"]}]}

this file will be named according to the Output Prefix given by the user, result1.json in this case.

Dependencies:

To run the program specify the correct arguments

python 3 jsonmerge_utils.py -ip tes -op result -maxFileSize 600 -dir_path /Users/mljbrackett/Documents/jsonmergeutils/SampleJsonfiles -log_level INFO

The command above runs the program as such:

The output log after a successful run will look like:

from

merge = Merge(path_creator(args.data_dir, args.input_prefix),
                      path_creator(args.data_dir, args.output_prefix),
                      args.max_file_size)

to (Copy the code below and replace the code from line 101-103)

merge = Merge(path_creator(args.data_dir, args.input_prefix),
                      path_creator(output_folder(args.data_dir), args.output_prefix),
                      args.max_file_size)

The program will create a ‘MergedFiles’ folder and store all the merged files in it.

Algorithmic complexity O(n):

Limitations: