Data Schema
Data Schema Format
Data schema is written in JSON-format. You cannot change the order of elements.
Example:
{
"SchemaVersion": "1.0",
"TableDefinition": {
"Columns": [
{ "Name": "col1", "DataType": { "DataType": "number" } }
]
},
"DataSpecification": {
"FormatType": "csv",
"DeliveryType": "file",
"DataOptions": {
"Header": true,
"Encoding":"utf-8",
"FieldDelimiter":",",
"NullString":"NA"
}
}
}
SchemaVersion
Specifies the version of schema.
Only "1.0"
is supported.
TableDefinition
Specifies the table structure.
Columns
List of columns of the table.
- Name: the column name (must starts with alphabet and contains only alphabet, numeric, or underscore)
- OriginalName: the original name of the column
- DataType: the data type
DataSpecification
Specifies the input data.
FormatType
Specifies the file format.
Only "CSV"
is supported.
DeliveryType
Specifies the file delivery.
"file"
and "collection"
are supported.
DataOptions
Specifies the options for the file reader.
Key | Value | Note |
---|---|---|
Header | true or false |
|
Encoding | "utf-8" |
It accepts not only "utf-8" but also popular encoding names. Non-UTF-8 encodings may work, but they are not supported. |
FieldDelimiter | string value | |
NullString | string value | |
PartitionUnit | "year" , "month" , "day" , or "hour" |
Set only when DeliveryType is collection. |
PartitionColumnName | string value | Set only when DeliveryType is collection. |