Documentation ata

README.md

@@ -3,28 +3,26 @@
 
 # Zendro
 
-Zendro is a software tool to quickly create a data warehouse tailored to your specifications. You tell Zendro what the structure of your data is, in the form of models, and where the data is or shall be stored. Zendro will then automatically create two standardized interfaces for your data models. Both interfaces provide access to the standard CRUD, create, read, update, and delete functions, available for each of the defined data models. One of the two interfaces is an intuitive graphical browser based single page application implemented in Google's standard material design. The other is an exhaustive application programming interface built with Facebook's efficient GraphQL framework, enabling a connection to your data warehouse from any programming language or data analysis pipeline with utmost ease, simply by sending HTTP requests to your GraphQL server. Data can be distributed over several databases and servers without losing the relationships between your data records, even if they are not stored in the same place.
+Zendro is a software tool to quickly create a data warehouse tailored to your specifications. You tell Zendro what the structure of your data is, in the form of models, and where the data is or shall be stored. Zendro will then automatically create two standardized interfaces for your data models. Both interfaces provide access to the standard CRUD, create, read, update, and delete functions, available for each of the defined data models. One of the two interfaces is an intuitive graphical browser based single page application (SPA) implemented in Google's standard [material design](https://material.io/). The other is an exhaustive application programming interface (API) built with Facebook's efficient GraphQL framework, enabling a connection to your data warehouse from any programming language or data analysis pipeline with utmost ease, simply by sending HTTP requests to your GraphQL server. Data can be distributed over several databases and servers without losing the relationships between your data records, even if they are not stored in the same place.
 
-Zendro consists of two main components, backend and frontend. The backend component has its [base project](https://github.com/ScienceDb/graphql-server) and a [code generator](https://github.com/ScienceDb/graphql-server-model-codegen). The frontend of SPA (Single Page Application) also has its [base project](https://github.com/ScienceDb/single-page-app) and a [code generator](https://github.com/ScienceDb/single-page-app-codegen).
-See the guides below on how to use Zendro.
-
-Also find Zendro-dev on [github](https://github.com/Zendro-dev).  
-
-If you have any questions or comments, please don't hesitate to contact us via an issue [here](https://github.com/Zendro-dev/Zendro-dev.github.io/issues). Tag your issue as a question and we will try to answer as quick as possible.
+Zendro consists of two main components, backend and frontend. The backend component has its [base project](https://github.com/ScienceDb/graphql-server) and a [code generator](https://github.com/ScienceDb/graphql-server-model-codegen). The frontend component (SPA) also has its [base project](https://github.com/ScienceDb/single-page-app) and a [code generator](https://github.com/ScienceDb/single-page-app-codegen). All of the component repositories mentioned here are also pinned under find [Zendro-dev](https://github.com/Zendro-dev).  
+The following instructions will show you how to use Zendro.
 
 [<p align="center"><img src="./figures/quick.png" /></p>](quickstart.md) 
 
 [<p align="center"><img src="./figures/gettingstarted.png"/></p>](setup_root.md)
 
 ### HOW-TO GUIDES:
-* [How to define data models: for developers](setup_data_scheme.md). Detailed technical specifications on how to define data models for Zendro, aimed at software developers and system administrators.
-* [How to define data models: for non-developers](non-developer_documentation.md). A brief, illustrated guide of data model specifications, data formatting and data uploading options, aimed at data modelers or managers to facilitate collaboration with developers.
-* [How to setup a distributed cloud of zendro nodes](ddm.md). A brief guide, aimed at software developers and system administrators, on how to use Zendros distributed-data-models.
-* [How to use Zendro command line interface (CLI)](zendro_cli.md). A tutorial of Zendro CLI, aimed at software developers.
-* [How to query and extract data](fromGraphQlToR.html). A concise guide on how to use the Zendro API from R to extract data and perform queries, aimed at data managers or data scientists.
-* [How to setup Authentication / Authorization](oauth.md). A concise guide on how to use and setup the Zendro authorization / authentication services.
+* [How to define data models (for developers): ](setup_data_scheme.md). Detailed technical specifications on how to define data models for Zendro, aimed at software developers and system administrators.
+* [How to define data models (for non-developers): ](non-developer_documentation.md). A brief, illustrated guide of data model specifications, data formatting and data uploading options, aimed at data modelers or managers to facilitate collaboration with developers.
+* [How to setup a distributed cloud of zendro nodes: ](ddm.md) A brief guide, aimed at software developers and system administrators, on how to use Zendros distributed-data-models.
+* [How to use Zendro command line interface (CLI): ](zendro_cli.md) A tutorial of Zendro CLI, aimed at software developers.
+* [How to query and extract data using R: ](fromGraphQlToR.html) A concise guide on how to use the Zendro API from R to extract data and perform queries, aimed at data managers or data scientists.
+* [How to setup Authentication / Authorization: ](oauth.md) A concise guide on how to use and setup the Zendro authorization / authentication services.
 * [API documentation](api_root.md).
 
+If you have any questions or comments, please don't hesitate to contact us via an issue [here](https://github.com/Zendro-dev/Zendro-dev.github.io/issues). Tag your issue as a question and we will try to answer as quick as possible.
+
 ### REPOSITORIES:
 
 * [GraphQL server](https://github.com/ScienceDb/graphql-server)
@@ -41,7 +39,7 @@ Francisca Acevedo<sup>1</sup>, Vicente Arriaga<sup>1</sup>, Katja Dohm<sup>3</su
 ##### Author affiliations
 1. CONABIO - Comisión Nacional para el Conocimiento y Uso de la Biodiversidad, México
 2. Forschungszentrum Jülich - Germany
-3. auticon - www.auticon.com
+3. Auticon - www.auticon.com
 4. InterTech - www.intertech.de
 
 #### Zendro author contributions

non-developer_documentation.md

@@ -44,7 +44,7 @@ To translate the conceptual diagram into JSON, we need follow the [JSON specific
             "reverseAssociation": "taxon_information", // The name of the reverse association defined in the specimen model
             "target": "specimen",
             "targetKey": "specimen_id",
-            "keyIn": "taxon",
+            "keyIn": "specimen",
             "targetStorageType": "sql",
             "label": "common_name",
         },

quickstart.md

@@ -12,6 +12,7 @@ If you want to know more about Zendro or a detailed explanation on how to set up
  * [NodeJS](https://nodejs.org/en/) version 14+ is required.
  * [docker](https://docs.docker.com/get-docker/)
  * [docker-compose](https://docs.docker.com/compose/install/#install-compose)
+ * [yarn](https://www.npmjs.com/package/yarn)
  <br/><br/>
 
 * * *

setup_data_scheme.md

@@ -2,26 +2,34 @@
 <br/>
 
 # Table of Contents
-* TOC
-{:toc}
+- [Data Models](#data-models)
+  - [JSON Specs](#json-specs)
+  - [Supported Data Types](#supported-data-types)
+  - [Associations Spec](#associations-spec)
+  - [The Resolver Layer and the Model Layer](#the-resolver-layer-and-the-model-layer)
+  - [The code generator at work](#the-code-generator-at-work)
+  - [Authorization-checks and record limits](#authorization-checks-and-record-limits)
+  - [Pagination types](#pagination-types)
+  - [Custom Validator Function for AJV](#custom-validator-function-for-ajv)
+  - [Data Loader](#data-loader)
 
 # Data Models
 
-For each one of the data sets that you want to include in the project you will need to describe the data model. This description should include its relations or associations with any other model. The description should be placed in a json file following the [json specs](#json-specs) for this purpose. You will need to store all these json files in a single folder. Another limitation is that each model should have a unique name independently of its type. From now on, in this document, we will assume that all json files for each one of your data models will be stored in the directory `/your-path/json-files`
+For each one of the data sets that you want to include in the project you will need to describe the data model. This description can include its relations or associations with any other model. The description should be placed in a json file following the [json specs](#json-specs) for this purpose. You will need to store all these json files in a single folder. Another limitation is that each model should have a unique name independently of its type. From now on, in this folder, we will assume that all json files for each one of your data models will be stored in the directory `/your-path/json-files`
 
 ## JSON Specs
 
-Each json file describes one and only one model. However, one model can reference another model using the associations mechanism described below.
+Each json file describes one and only one data model. However, one data model can reference another data model using the associations mechanism described below.
 
-For a complete description of each model we need to specify the following fields in the json file:
+For a complete description of each data model we need to specify the following fields in the json file:
 
 Name | Type | Description
 ------- | ------- | --------------
-*model* | String | Name of the model (it is recommended to use snake_case naming style to obtain nice names in the auto-generated GraphQL API). The string here can not contain spaces.
-*model_name_in_storage* | String | The name of the model in the storage itself. E.g the table-name in relation dbs, the collection in mongodb, the node in neo4j, etc. By default Zendro uses the lowercase pluralized *model* property. 
+*model* | String | Name of the data model (it is recommended to use snake_case naming style to obtain nice names in the auto-generated GraphQL API). The string here can not contain spaces.
+*model_name_in_storage* | String | The name of the data model in the storage itself. E.g the table-name in relation dbs, the collection in mongodb, the node in neo4j, etc. By default Zendro uses the lowercase pluralized *model* property. 
 *database* | String | Name of the database connection as a key defined in [`data_models_storage_config.json`](https://github.com/Zendro-dev/graphql-server/blob/master/config/data_models_storage_config.json). If this field is not defined, the database connection used will be `default-<storageType>`.
-*storageType* | String | Type of storage where the model is stored. So far can be one of: <ul> <li>  __sql__ for local relational databases supported by [sequelize](#http://docs.sequelizejs.com/) such as PostgreSql/MySql etc. </li>  <li>__generic__ for any database that your project would connect remotely.</li>  <li>__zendro\_server__ for models stored in any other instance created with zendro-tools.</li><li>  __cassandra__ for local cassandra databases supported by datastax node [cassandra-driver](https://docs.datastax.com/en/developer/nodejs-driver/4.6/). Refer to [cassandra storageType documentation](cassandra_storageType.md) for cassandra specific restrictions.</li></ul>
-*url* | String | This field is only mandatory for __zendro\_server__ stored models. Indicates the url where the zendro server storing the model is runnning.
+*storageType* | String | Type of storage where the data model is stored. So far can be one of: <ul> <li>  __sql__ for local relational databases supported by [sequelize](#http://docs.sequelizejs.com/) such as PostgreSql/MySql etc. </li>  <li>__generic__ for any database that your project would connect remotely.</li>  <li>__zendro\_server__ for data models stored in any other instance created with zendro-tools.</li><li>  __cassandra__ for local cassandra databases supported by datastax node [cassandra-driver](https://docs.datastax.com/en/developer/nodejs-driver/4.6/). Refer to [cassandra storageType documentation](cassandra_storageType.md) for cassandra specific restrictions.</li></ul>
+*url* | String | This field is only mandatory for __zendro\_server__ stored data models. Indicates the url where the zendro server storing the data model is runnning.
 *attributes* | Object |  The key of each entry is the name of the attribute and there are two options for the value . It can be either a string indicating the type of the attribute or an object where the user indicates the type of the attribute(in the _type_ field) together with an attribute's description (in the _description_ field). See the [table](#supported-data-types) below for allowed types. Example of option one: ```{ "attribute1" : "String", "attribute2: "Int" }``` Example of option two: ``` { "attribute1" : {"type" :"String", "description": "Some description"}, "attribute2: "Int ```
 *associations* | Object | The key of each entry is the name of the association and the value should be an object describing the corresponding association. See [Associations Spec](#associations-spec) section below for details.
 *indices* | [String] |  Names of attributes for generating corresponding indices.
@@ -75,7 +83,7 @@ name | Type | Description
 It's important to notice that when a model involves a foreign key for the association, this key should be explicitly written into the attributes field of the given local model. Although, foreign keys will be available for the user only as readable attributes, for editing this attributes we offer the possibility as part of the API, please see [this](api_graphql.md#extra-mutation-fields-to-update-or-create-associations) section for more info.
 To store to-many associations (many-to-many or one-to-many) via foreign keys Zendro offers to store the foreign keys in arrays. In this case the model will have an array attribute which will store ids from the associated records.
 
-#### single-end foreign keys
+#### Single-end foreign keys
 Storing the foreign keys on a single end of the association means that only one of the two associated data-models holds the foreign-key attribute. Storing the keys in that way guarantees fast write actions and avoids error prone operations of writing multiple records to update any association. It also requires less storage space, but can become slow to read and search, especially in a distributed context, where the associated records could be distributed over multiple servers.
 
 To define single-end foreign key associations the following arguments need to be added:
@@ -130,7 +138,7 @@ Examples:
   }
 }
 ```
-#### paired-end foreign keys
+#### Paired-end foreign keys
 Storing the association via paired-end foreign keys means that both associated data-models contain a reference (foreign key) to the associated records. Storing the keys in that way guarantees read and search efficiency, especially in a distributed context, at the cost of time and storage-space when handling write actions. Since the keys are stored at both ends the information needs to be updated at both ends as well, which is slower and more prone to errors.
 
 Many-to-many associations can be stored via paired-end associations. In this case both models will hold an array attribute which will store ids from the associated records. These two attributes will be described in the association as `sourceKey` and `targetKey`.
@@ -313,7 +321,7 @@ Example:
 ```
 
 
-### generic associations
+### Generic associations
 To generate a generic association the `generic` implementation type can be used. This will genereate code stubs in the models for the user specific implementation of resolving the management of the association.
 
 ### Differences between backend and Frontend (GUI)