Common functionality for the data processing, mostly concerning the locale independent structured data files reading and writing.
This library implements common functionality for the data processing, mostly concerning the files collections synchronization and batch operations, work flow automation and structured data files reading and writing. All implemented functionality is generic without concerns about specifics of proprietary or highly specialized file formats or details of the actual data processing.
Concerning the files collections synchronization specifically and general files operations the following functionality is provided:
Clone the official repository into your local workspace / project folder:
$ git clone <repository_path> "your project folder"
Initialize the UML templates submodule
$ cd "your project folder"/fsio_lib/Documentation/UML/Templates
$ git submodule init
Download the content of the UML templates submodule
$ git submodule update --recursive --remote
Note: You do not have to intialize and download the UML templates submodule if you do not plan to update the library’s documentation. They are not required for the functionality of the library iteself.
Implements functions for mapping the content of a nested C struct (or Pascal record) like objects, nested dictionaries, nested sequences or even XML representation objects (xml.etree.ElementTree.Element class) onto another object of one of these types using defined templates.
Implements functions to import modules or objects from modules dynamically, i.e. using their string names at the runtime.
Implements ‘safe/smart’ copy / move / rename / delete file operations, which intercept OS and I/O related exceptions raised during operations, which are logged into console and / or file and returned as the result of an operation.
Implements functions for locale independent writing of tabulated data ASCII text files (TSV format) concerning the new line, decimal separator and delimiter conventions as well as use of usual spaces instead of TABs for the columns separation. Also implements function, which forces specific new line convention.
Implements a number of functions for the maintenance of a file system, including recursive removal of the duplicating files, recursive folder’s renaming, recursive deletion of empty sub-folders, pulling of the non-present files from another folder, etc.
Implements singleton classes for parsing TSV, XML and JSON files with the data mapping onto specified class instances using the specified templates. Also defines generic parsing functions, which automatically select the required parsing class.
All documentation can be found in the Documentation folder (see Index), and it is written using Markdown format.