1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
|
|Linux Build Status| |Code coverage| |Documentation Status| |CII Best Practices|
.. |Linux Build Status| image:: https://github.com/common-workflow-language/schema_salad/actions/workflows/ci-tests.yml/badge.svg?branch=main
:target: https://github.com/common-workflow-language/schema_salad/actions/workflows/ci-tests.yml
.. |Code coverage| image:: https://codecov.io/gh/common-workflow-language/schema_salad/branch/main/graph/badge.svg
:target: https://codecov.io/gh/common-workflow-language/schema_salad
.. |Documentation Status| image:: https://readthedocs.org/projects/schema-salad/badge/?version=latest
:target: https://schema-salad.readthedocs.io/en/latest/?badge=latest
:alt: Documentation Status
.. |CII Best Practices| image:: https://bestpractices.coreinfrastructure.org/projects/1867/badge
:target: https://bestpractices.coreinfrastructure.org/projects/1867
Schema Salad
------------
Salad is a schema language for describing JSON or YAML structured
linked data documents. Salad schema describes rules for
preprocessing, structural validation, and hyperlink checking for
documents described by a Salad schema. Salad supports rich data
modeling with inheritance, template specialization, object
identifiers, object references, documentation generation, code
generation, and transformation to RDF_. Salad provides a bridge
between document and record oriented data modeling and the Semantic
Web.
The Schema Salad library is Python 3.6+ only.
Installation
------------
::
pip3 install schema_salad
If you intend to use the `schema-salad-tool --codegen=python` feature, please
include the `[pycodegen]` extra::
pip3 install schema_salad[pycodegen]
To install from source::
git clone https://github.com/common-workflow-language/schema_salad
cd schema_salad
pip3 install .
# or pip3 install .[pycodegen] if needed
Commands
--------
Schema salad can be used as a command line tool or imported as a Python module::
$ schema-salad-tool
usage: schema-salad-tool [-h] [--rdf-serializer RDF_SERIALIZER] [--skip-schemas]
[--strict-foreign-properties] [--print-jsonld-context]
[--print-rdfs] [--print-avro] [--print-rdf] [--print-pre]
[--print-index] [--print-metadata] [--print-inheritance-dot]
[--print-fieldrefs-dot] [--codegen language] [--codegen-target CODEGEN_TARGET]
[--codegen-examples directory] [--codegen-package dotted.package]
[--codegen-copyright copyright_string] [--print-oneline]
[--print-doc] [--strict | --non-strict]
[--verbose | --quiet | --debug] [--only ONLY] [--redirect REDIRECT]
[--brand BRAND] [--brandlink BRANDLINK] [--brandstyle BRANDSTYLE]
[--brandinverse] [--primtype PRIMTYPE] [--version]
[schema] [document]
$ python
>>> import schema_salad
Validate a schema::
$ schema-salad-tool myschema.yml
Validate a document using a schema::
$ schema-salad-tool myschema.yml mydocument.yml
Generate HTML documentation::
$ schema-salad-tool --print-doc myschema.yml > myschema.html
$ # or
$ schema-salad-doc myschema.yml > myschema.html
Get JSON-LD context::
$ schema-salad-tool --print-jsonld-context myschema.yml mydocument.yml
Convert a document to JSON-LD::
$ schema-salad-tool --print-pre myschema.yml mydocument.yml > mydocument.jsonld
Generate Python classes for loading/generating documents described by the schema
(Requires the `[pycodegen]` extra)::
$ schema-salad-tool --codegen=python myschema.yml > myschema.py
Display inheritance relationship between classes as a graphviz 'dot' file and
render as SVG::
$ schema-salad-tool --print-inheritance-dot myschema.yml | dot -Tsvg > myschema.svg
Codegen Examples
----------------
These are helpful to see how to use the output of `schema-salad-tool --codegen`
in different languages for loading and/or creating/editing/saving objects,
using the `CWL v1.2 schema <https://github.com/common-workflow-language/cwl-v1.2/blob/1.2.1_proposed/CommonWorkflowLanguage.yml>`_
as an example.
+-------------+---------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Language | Repository | Serialization Example | Deserialization Example |
+=============+=========================================================+======================================================================================================================================================+============================================================================================================================================================================+
| Python | https://github.com/common-workflow-language/cwl-utils/ | `create_cwl_from_objects.py <https://github.com/common-workflow-language/cwl-utils/blob/main/create_cwl_from_objects.py>`_ | `load_document() <https://github.com/common-workflow-language/cwl-utils/blob/main/cwl_utils/parser/__init__.py#L93>`_ |
+-------------+---------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Java | https://github.com/common-workflow-language/cwljava/ | (Not yet implemented) | `PackedWorkflowClassTest.java <https://github.com/common-workflow-language/cwljava/blob/cwl-1.2.0/src/test/java/org/w3id/cwl/cwl1_2/utils/PackedWorkflowClassTest.java>`_ |
+-------------+---------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| TypeScript | https://github.com/common-workflow-lab/cwl-ts-auto | `Creating, editing, and saving CWL docs with TypeScript <https://github.com/common-workflow-lab/cwl-ts-auto#creating-editing-and-saving-documents>`_ | `Loading CWL documents with TypeScript <https://github.com/common-workflow-lab/cwl-ts-auto#loading-documents>`_ |
+-------------+---------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| .Net | https://github.com/common-workflow-lab/CWLDotNet | `Creating, editing, and saving CWL docs with .Net <https://github.com/common-workflow-lab/CWLDotNet#creating-editing-and-serializing-documents>`_ | `Loading CWL documents with .Net <https://github.com/common-workflow-lab/CWLDotNet#loading-documents>`_ |
+-------------+---------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| C++ | https://github.com/common-workflow-lab/cwl-cpp-auto | `cwl_output_example.cpp <https://github.com/common-workflow-lab/cwl-cpp-auto/blob/main/cwl_output_example.cpp>`_ | (Not yet implemented) |
+-------------+---------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| D | https://github.com/common-workflow-lab/cwl-d-auto | `How to use <https://github.com/common-workflow-lab/cwl-d-auto#how-to-use>`_ | `How to use <https://github.com/common-workflow-lab/cwl-d-auto#how-to-use>`_ |
+-------------+---------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Quick Start
-----------
Let's say you have a 'basket' record that can contain items measured either by
weight or by count. Here's an example::
basket:
- product: bananas
price: 0.39
per: pound
weight: 1
- product: cucumbers
price: 0.79
per: item
count: 3
We want to validate that all the expected fields are present, the
measurement is known, and that "count" cannot be a fractional value.
Here is an example schema to do that::
- name: Product
doc: |
The base type for a product. This is an abstract type, so it
can't be used directly, but can be used to define other types.
type: record
abstract: true
fields:
product: string
price: float
- name: ByWeight
doc: |
A product, sold by weight. Products may be sold by pound or by
kilogram. Weights may be fractional.
type: record
extends: Product
fields:
per:
type:
type: enum
symbols:
- pound
- kilogram
jsonldPredicate: '#per'
weight: float
- name: ByCount
doc: |
A product, sold by count. The count must be a integer value.
type: record
extends: Product
fields:
per:
type:
type: enum
symbols:
- item
jsonldPredicate: '#per'
count: int
- name: Basket
doc: |
A basket of products. The 'documentRoot' field indicates it is a
valid starting point for a document. The 'basket' field will
validate subtypes of 'Product' (ByWeight and ByCount).
type: record
documentRoot: true
fields:
basket:
type:
type: array
items: Product
You can check the schema and document in schema_salad/tests/basket_schema.yml
and schema_salad/tests/basket.yml::
$ schema-salad-tool basket_schema.yml basket.yml
Document `basket.yml` is valid
Documentation
-------------
See the specification_ and the metaschema_ (salad schema for itself). For an
example application of Schema Salad see the Common Workflow Language_.
Rationale
---------
The JSON data model is an popular way to represent structured data. It is
attractive because of it's relative simplicity and is a natural fit with the
standard types of many programming languages. However, this simplicity comes
at the cost that basic JSON lacks expressive features useful for working with
complex data structures and document formats, such as schemas, object
references, and namespaces.
JSON-LD is a W3C standard providing a way to describe how to interpret a JSON
document as Linked Data by means of a "context". JSON-LD provides a powerful
solution for representing object references and namespaces in JSON based on
standard web URIs, but is not itself a schema language. Without a schema
providing a well defined structure, it is difficult to process an arbitrary
JSON-LD document as idiomatic JSON because there are many ways to express the
same data that are logically equivalent but structurally distinct.
Several schema languages exist for describing and validating JSON data, such as
JSON Schema and Apache Avro data serialization system, however none
understand linked data. As a result, to fully take advantage of JSON-LD to
build the next generation of linked data applications, one must maintain
separate JSON schema, JSON-LD context, RDF schema, and human documentation,
despite significant overlap of content and obvious need for these documents to
stay synchronized.
Schema Salad is designed to address this gap. It provides a schema language
and processing rules for describing structured JSON content permitting URI
resolution and strict document validation. The schema language supports linked
data through annotations that describe the linked data interpretation of the
content, enables generation of JSON-LD context and RDF schema, and production
of RDF triples by applying the JSON-LD context. The schema language also
provides for robust support of inline documentation.
.. _JSON-LD: http://json-ld.org
.. _Avro: http://avro.apache.org
.. _metaschema: https://github.com/common-workflow-language/schema_salad/blob/main/schema_salad/metaschema/metaschema.yml
.. _specification: http://www.commonwl.org/v1.2/SchemaSalad.html
.. _Language: https://github.com/common-workflow-language/cwl-v1.2/blob/v1.2.0/CommandLineTool.yml
.. _RDF: https://www.w3.org/RDF/
|
