Data Schema

Data Schema Format

Data schema is written in JSON-format. You cannot change the order of elements.

Example:

{
  "SchemaVersion": "1.0",
  "TableDefinition": {
    "Columns": [
      { "Name": "col1", "DataType": { "DataType": "number" } }
    ]
  },
  "DataSpecification": {
    "FormatType": "csv",
    "DeliveryType": "file",
    "DataOptions": {
      "Header": true,
      "Encoding":"utf-8",
      "FieldDelimiter":",",
      "NullString":"NA"
    }
  }
}

SchemaVersion

Specifies the version of schema.

Only "1.0" is supported.

TableDefinition

Specifies the table structure.

Columns

List of columns of the table.

  • Name: the column name (must starts with alphabet and contains only alphabet, numeric, or underscore)
  • OriginalName: the original name of the column
  • DataType: the data type

DataSpecification

Specifies the input data.

FormatType

Specifies the file format.

Only "CSV" is supported.

DeliveryType

Specifies the file delivery.

"file" and "collection" are supported.

DataOptions

Specifies the options for the file reader.

Key Value Note
Header true or false
Encoding "utf-8" It accepts not only "utf-8" but also popular encoding names. Non-UTF-8 encodings may work, but they are not supported.
FieldDelimiter string value
NullString string value
PartitionUnit "year", "month", "day", or "hour" Set only when DeliveryType is collection.
PartitionColumnName string value Set only when DeliveryType is collection.