File: supported_types.md

package info (click to toggle)
typedload 2.40-1
  • links: PTS, VCS
  • area: main
  • in suites: sid
  • size: 844 kB
  • sloc: python: 3,322; makefile: 145
file content (572 lines) | stat: -rw-r--r-- 13,981 bytes parent folder | download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
Supported types
===============

None
----

```python
typedload.load(obj, None)
```

It will either return a None or fail.

This is normally used to handle unions such as `int | None` rather than by itself.

Basic types
-----------

By default: `{int, bool, float, str, NONETYPE}`

Those types are the basic building blocks and no operations are performed on them.

*NOTE*: If `basiccast=True` (the default) casting between them can still happen.

```python
In : typedload.load(1, float)
Out: 1.0

In : typedload.load(1, str)
Out: '1'

In : typedload.load(1, int)
Out: 1

In : typedload.load(1, float, basiccast=False)
Exception: TypedloadValueError

In : typedload.load(1, bool, basiccast=False)
Exception: TypedloadValueError
```

The `basictypes` set can be tweaked.

```python
In : typedload.load(1, bytes, basictypes={bytes, int})
Out: b'\x00'

In : typedload.load(1, int, basictypes={bytes, int})
Out: 1
```

typing.Literal
--------------

```python
typedload.load(obj, Literal[1])
typedload.load(obj, Literal[1,2,3])
```

Succeeds only if obj equals one of the allowed values.

This is normally used in objects, to decide the correct type in a `Union`.

It is very common to use Literal to disambiguate objects in a Union. [See example](examples.md#object-type-in-value)

*This is very fast, because typedload will internally use the `Literal` values to try the best type in the union first.*

enum.Enum
---------

```python
class Flags(Enum):
    NOVAL = 0
    YESVAL = 1

In : typedload.load(1, Flags)
Out: <Flags.YESVAL: 1>


```

Load values from an Enum, when dumping the value is used.

list
----

```python
In : typedload.load([1, 2, 3], list[int])
Out: [1, 2, 3]

In : typedload.load([1.1, 2, '3'], list[int])
Out: [1, 2, 3]

In : typedload.load([1.1, 2, '3'], list[int], basiccast=False)
Exception: TypedloadValueError
```

Load an iterable into a list object.

Always dumped as a list.

tuple
-----

Always dumped as a list.

### Finite size tuple

```python
In : typedload.load([1, 2, 3], tuple[int, float])
Out: (1, 2.0)

# Be more strict and fail if there is more data than expected on the iterator
In : typedload.load([1, 2, 3], tuple[int, float], failonextra=True)
Exception: TypedloadValueError
```

### Infinite size tuple

```python
In : typedload.load([1, 2, 3], tuple[int, ...])
Out: (1, 2, 3)
```

Uses Ellipsis (`...`) to indicate that the tuple contains an indefinite amount of items of the same size.

dict
----

```python
In : typedload.load({1: '1'}, dict[int, Path])
Out: {1: PosixPath('1')}

In : typedload.load({1: '1'}, dict[int, str])
Out: {1: '1'}

In : typedload.load({'1': '1'}, dict[int, str])
Out: {1: '1'}

In : typedload.load({'1': '1'}, dict[int, str], basiccast=False)
Exception: TypedloadValueError

class A(NamedTuple):
    y: str='a'

In : typedload.load({1: {}}, dict[int, A], basiccast=False)
Out: {1: A(y='2')}

```

Loads a dictionary, making sure that the types are correct.

Objects
-------

* typing.NamedTuple
* dataclasses.dataclass
* attr.s

```python
class Point2d(NamedTuple):
    x: float
    y: float

class Point3d(NamedTuple):
    x: float
    y: float
    z: float

@attr.s
class Polygon:
    vertex: list[Point2d] = attr.ib(factory=list, metadata={'name': 'Vertex'})

@dataclass
class Solid:
    vertex: list[Point3d] = field(default_factory=list)
    total: int = field(init=False)

    def __post_init__(self):
        self.total = 123 # calculation here

In : typedload.load({'Vertex':[{'x': 1,'y': 1}, {'x': 2,'y': 2},{'x': 3,'y': 3}]}, Polygon)
Out: Polygon(vertex=[Point2d(x=1.0, y=1.0), Point2d(x=2.0, y=2.0), Point2d(x=3.0, y=3.0)])

In : typedload.load({'vertex':[{'x': 1,'y': 1,'z': 1}, {'x': 2,'y': 2, 'z': 2},{'x': 3,'y': 3,'z': 3}]}, Solid)
Out: Solid(vertex=[Point3d(x=1.0, y=1.0, z=1.0), Point3d(x=2.0, y=2.0, z=2.0), Point3d(x=3.0, y=3.0, z=3.0)], total=123)
```

They are loaded from dictionaries into those objects. `failonextra` when set can generate exceptions if more fields than expected are present.

When dumping they go back to dictionaries. `hide_default` defaults to True, so all fields that were equal to the default will not be dumped.

### attrs converters

Attrs fields can have a converter function associated.

If this is the case, typedload will ignore the assigned type, inspect the type hints of the converter function, and assign the type of the parameter of the converter as type. If the function is not typed, `Any` will be used.

This can be useful when the data format has been changed in a more complex way than just adding a few extra fields. Then the converter function can be used to do the necessary conversions for the old data format.

#### Examples

```python
@attr.s
class A:
    x: int = attr.ib(converter=str) # x has a converter that just calls str()

In : load({'x': [1]}, A)
Out: A(x='[1]')

# In this case the int type for x was completely ignored, because a converter is defined
# The str() function does not define type hints, so Any is used
# So the list [1] is passed as is to the constructor of A() which calls str() on it to convert it
```


```python
@attr.s
class Old:
    oldfield: int = attr.ib()

@attr.s
class New:
    newfield: int = attr.ib()

def conv(p: Old | New) -> New:
    # The type hinting necessary to tell typedload what to do
    # Without hinting it would just pass the dictionary directly
    if isinstance(p, New):
        return p
    return New(p.oldfield)

@attr.s
class Outer:
    '''
    Our old data format was using the Old class, but
    we now use the New class.

    The converter returns a New instance from an Old instance.
    '''
    inner: New = attr.ib(converter=conv)

# Calling load with the new data format, returns a New class
In : load({'inner': {'newfield':3}}, Outer)
Out: Outer(inner=New(newfield=3))
# Loading with the old data format, still returns a New class
In : load({'inner': {'oldfield':3}}, Outer)
Out: Outer(inner=New(newfield=3))
```

Forward references
------------------

A forward reference is when a type is specified as a string instead of as an object:

```python
a: ObjA = ObjA()
a: 'ObjA' = ObjA()
```

The 2nd generates a forward reference, that is, a fake type that is really hard to resolve.

The current strategy for typedload is to cache all the names of the types it encounters and use this cache to resolve the names.

In alternative, it is possible to use the `frefs` dictionary to manually force resolution for a particular type.

Python `typing` module offers some ways to resolve those types which are not used at the moment because they are slow and have strong limitations.

Python developers want to turn every type annotation into a forward reference, for speed reasons. This was supposed to come in 3.10 but has been postponed. So for the moment there is little point into working on this very volatile API.


typing.Union
------------

A union means that a value can be of more than one type.

If the passed value is of a `basictype` that is also present in the Union, the value will be returned.

Otherwise, basictype values are evaluated last. This is to avoid that a Union containing a `str` will catch more than it should.

```python3
typedload.load(data, int | str)
```


### Tagged unions

If all the types within the union have a field of Literal type, that will be used to quickly inspect the value and decide which type to use.

Unlike other libraries, no manual action needs to be taken, besides having the fields with the Literal type in each member of the union.

```python3
@dataclass
class A:
    type: Literal['A']
    ...

@dataclass
class B:
    type: Literal['B']
    ...

# It will inspect the data and try the correct type directly
typedload.load(data, A | B)
```

### Optional

A typical case is when using Optional values

```python
In : typedload.load(3, int | None)
Out: 3

In : typedload.load(None, int | None)
Out: None
```

### Ambiguity

Ambiguity can sometimes be fixed by enabling `failonextra` or disabling `basiccast`.

```python
Point2d = tuple[float, float]
Point3d = tuple[float, float, float]

# This is not what we wanted, the 3rd coordinate is lost
In : typedload.load((1,1,1), Point2d | Point3d)
Out: (1.0, 1.0)

# Make the loading more strict!
In : typedload.load((1,1,1), Point2d | Point3d, failonextra=True)
Out: (1.0, 1.0, 1.0)
```

But in some cases it cannot be simply solved, when the types in the Union are too similar. In this case the only solution is to rework the codebase.

```python
# A casting must be done, str was chosen, but could have been int
In : typedload.load(1.1, str | int)
Out: '1.1'


class A(NamedTuple):
    x: int=1

class B(NamedTuple):
    y: str='a'

# Both A and B accept an empty constructor
In : typedload.load({}, A | B)
Out: A(x=1)
```

#### Finding ambiguity

Typedload can't solve certain ambiguities, but setting `uniondebugconflict=True` will help detect them.

```python
In : typedload.load({}, A | B, uniondebugconflict=True)
TypedloadTypeError: Value of dict could be loaded into typing.Union[__main__.A, __main__.B] multiple times
```

So this setting can be used to find ambiguities and manually correct them.

*NOTE*: The setting slows down the loading of unions, so it is recommended to use it only during tests or when designing the data structures, but not in production.


typing.TypedDict
----------------

```python
class A(TypedDict):
    val: str

In : typedload.load({'val': 3}, A)
Out: {'val': '3'}

In : typedload.load({'val': 3,'aaa':2}, A)
Out: {'val': '3'}

In : typedload.load({'val': 3,'aaa':2}, A, failonextra=True)
Exception: TypedloadValueError
```

From dict to dict, but it makes sure that the types are as expected.

It also supports non-total TypedDict.

```python
class A(TypedDict, total=False):
    val: str

In : typedload.load({}, A)
Out: {}
```
### Required/NotRequired

**Required** and **NotRequired** can also be used.

```python
class A(TypedDict):
    val: str
    vol: NotRequired[int]

In : typedload.load({'val': 'a'}, A)
Out: {'val': 'a'}
```

```python
class A(TypedDict, total=False):
    val: str
    vol: Required[int]

In : typedload.load({'val': 'a', 'vol': 1}, A)
Out: {'val': 'a', 'vol': 1}
```

### ReadOnly

ReadOnly can be used, the effect is that the inner type gets used to typechecking and it is otherwise ignored.

set, frozenset
--------------

```python
In : typedload.load([1, 4, 99], set[float])
Out: {1.0, 4.0, 99.0}

In : typedload.load(range(12), set[int])
Out: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11}

In : typedload.load(range(12), frozenset[float])
Out: frozenset({0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0})
```

Loads an iterable inside a `set` or a `frozenset`.

Always dumped as a list.

typing.Any
----------

```python
typedload.load(obj, typing.Any)
```

This will just return `obj` without doing any check or transformation.

To work with `dump()`, `obj` needs to be of a supported type, or an handler is needed.

typing.NewType
--------------

```python
T = typing.NewType('T', str)
typedload.load('ciao', T)
```

Allows the use of NewType to define already handled types.

typing.TypeAliasType
--------------------

This was instroduced with *PEP 695: Type Parameter Syntax*, and is available since python 3.12.

in typedload it is possible to do this

```python
type number = int | float
typedload.load(3, number)
```

It is possible to use TypeAliasType as types of fields:

```python
type number = int | float

class A(NamedTuple):
    i: number

typedload.load({'i': 3}, A)
```

### attr

The feature works with attr if regular type annotations are used. But at this point converter functions using type aliases are not supported.

### Generics

The generics are not supported, because typedload needs an actual type to use, so `type Point[T] = tuple[T, T]` will not work. This is not a bug.

### Maturity

This feature is very new. There are test cases but since it hasn't been used in production yet, there might be missing features or issues.

String constructed
------------------

Loaders and dumpers have a set of `strconstructed`.

Those are types that accept a single `str` parameter in their constructor and have a `__str__` method that returns that parameter.

It is possible to add more by adding them to the `strconstructed` set.

The preset ones are:

### pathlib.Path

```python
In : typedload.load('/tmp/', Path)
Out: PosixPath('/tmp')

In : typedload.load('/tmp/file.txt', Path)
Out: PosixPath('/tmp/file.txt')
```

Loads a string as a `Path`; dumps it as a string.

### ipaddress.IPv*Address/Network/Interface

* `ipaddress.IPv4Address`
* `ipaddress.IPv6Address`
* `ipaddress.IPv4Network`
* `ipaddress.IPv6Network`
* `ipaddress.IPv4Interface`
* `ipaddress.IPv6Interface`

```python
In : typedload.load('10.1.1.3', IPv4Address)
Out: IPv4Address('10.1.1.3')
```

Loads a string as an one of those classes, and dumps as a string.

### uuid.UUID

* `uuid.UUID`

Loads a string as `UUID`; dumps it as a string.

argparse.Namespace
------------------

This is converted to a dictionary and can be loaded into NamedTuple/dataclass.

Dates
-----

### `datetime.timedelta`
Represented as a float of seconds.


### `datetime.date` `datetime.time` `datetime.datetime`

When loading, it is possible to pass a string in ISO 8601, or a list of ints that will be passed to the constructor.

When dumping, the default is to dump a list of ints, unless `isodates=True` is set in the dumper object, in which case an ISO 8601 string will be returned instead.

The format with the list of ints is deprecated and kept for backward compatibility. Everybody should use the ISO 8601 strings.

The format with the list of ints does not support timezones.

re.Pattern
----------

Loads a str or bytes as a compiled Pattern object by passing through re.compile.
When dumping gives back the original str or bytes pattern.

typedload.moretypes.HexRGB
--------------------------

This class loads and dumps as a string, but also validates that the string is a valid RGB colour in the form '#nnnnnn'.