1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259
|
`Hishel` provides powerful tools for improving your transports.
It analyzes your responses and saves them so they can be reused in the future.
It is very simple to integrate with your existing application; simply change the `Client` or `Transport` class that you are using.
## Clients and Transports
There are three ways to make the httpx library cacheable when working with it.
- Use the class provided by `Hishel` to completely replace `HTTPX's Client`.
- Simply use your existing httpx client along with `Hishel's transports`.
- Mock the httpx classes using the `hishel.install_cache` function.
It is always advised to use the second option because it is more reliable and adaptable.
!!! warning
Use the `hishel.install_cache` function only for experiments, and do not rely on the functionality provided by `hishel.install_cache`.
### Using the Clients
`Hishel` offers two classes for the first choice.
- `hishel.AsyncCacheClient` for `httpx.AsyncClient`
- `hishel.CacheClient` for `httpx.Client`
This implies that you can enable HTTP caching in your existing application by simply switching to the proper Client.
Examples:
```python
>>> import hishel
>>>
>>> with hishel.CacheClient() as client:
>>> client.get("https://example.com/cachable-endpoint")
>>> response = client.get("https://example.com/cachable-endpoint") # from the cache!
```
Asynchronous Example:
```python
>>> with hishel.AsyncCacheClient() as client:
>>> await client.get("https://example.com/cachable-endpoint")
>>> response = await client.get("https://example.com/cachable-endpoint") # from the cache!
```
!!! warning
The client classes that `Hishel` offers hide the constructor signature **in order to support all possible httpx versions.** This means that all httpx client fields are fully valid for those clients, but because they are hidden, **your IDE cannot suggest** which arguments you can pass. In other words, these classes merely use **\*args** and **\*\*kwargs** and add a few arguments for cache configuration.
This example also functions as long as the cache clients are fully compatible with the standard clients.
Example:
```python
client = hishel.CacheClient(
proxies={
"all://": "https://myproxy.com"
},
auth=("login", "password"),
follow_redirects=True,
http1=False,
http2=True
)
client.get("https://example.com")
```
### Specifying the Client storage
Sometimes you may need to select storage rather than filesystem, and this is how you do it.
```python
import hishel
storage = hishel.RedisStorage()
with hishel.CacheClient(storage=storage) as client:
client.get("https://example.com")
```
The responses are now saved in the [redis](https://redis.io/) database.
By default it will use...
- host: **localhost**
- port: **6379**.
Of course, you can explicitly set each configuration.
Example:
```python
import hishel
import redis
storage = hishel.RedisStorage(
client=redis.Redis(
host="192.168.0.85",
port=8081,
)
)
with hishel.CacheClient(storage=storage) as client:
client.get("https://example.com")
```
!!! note
Make sure `Hishel` has the redis extension installed if you want to use the Redis database.
``` shell
$ pip install hishel[redis]
```
### Using the Transports
It is always preferable to use transports that `Hishel` offers for more dependable and predictable behavior.
We advise you to read the [transports documentation](https://www.python-httpx.org/advanced/transports/) if you have never used `HTTPX's transports` before continuing.
We can divide the httpx library into two parts: the transports and the rest of the httpx library. Transports are the objects that are **actually making the request**.
For synchronous and asynchronous requests, `Hishel` offers two different transports.
- CacheTransport
- AsyncCacheTransport
`Hishel` always needs a transport to work on top of it, as long as he **respects the custom or third-party transports that are offered.**
Example:
```python
import hishel
import httpx
with httpx.HTTPTransport() as transport:
with hishel.CacheTransport(transport=transport) as cache_transport:
request = httpx.Request("GET", "https://example.com/cachable-endpoint")
cache_transport.handle_request(request)
response = cache_transport.handle_request(request) # from the cache!
```
#### Using the Transports with the Clients
If you have a transport, you can provide it to clients who will use it for underlying requests.
```python
import hishel
import httpx
cache_transport = hishel.CacheTransport(transport=httpx.HTTPTransport())
with httpx.Client(transport=cache_transport) as client:
client.get("https://example.com/cachable-endpoint")
response = client.get("https://example.com/cachable-endpoint") # from the cache
```
#### Specifying the Transport storage
In the same way that we can choose the storage for our clients, we can do the same for our transport.
```python
import hishel
storage = hishel.RedisStorage()
with httpx.HTTPTransport() as transport:
with hishel.CacheTransport(transport=transport, storage=storage) as cache_transport:
request = httpx.Request("GET", "https://example.com/cachable-endpoint")
cache_transport.handle_request(request)
```
#### Combining with the existing Transports
Assume you already have a custom transport adapted to your business logic that you use for all requests; this is how you can add the caching layer on top of it.
```python
import hishel
import httpx
from my_custom_transports import MyLovelyTransport
cache_transport = hishel.CacheTransport(transport=MyLovelyTransport())
with httpx.Client(transport=cache_transport) as client:
client.get("https://example.com/cachable-endpoint")
response = client.get("https://example.com/cachable-endpoint") # from the cache
```
### Using the Connection Pool
`Hishel` also provides caching support for the httpcore library, which handles all of the low-level network staff for httpx.
You may skip this section if you do not use [HTTP Core](https://github.com/encode/httpcore).
Example:
```python
import hishel
import httpcore
with httpcore.ConnectionPool() as pool:
with hishel.CacheConnectionPool(pool=pool) as cache_pool:
cache_pool.get("https://example.com/cachable-endpoint")
response = cache_pool.get("https://example.com/cachable-endpoint") # from the cache
```
#### Specifying the Connection Pool storage
In the same way that we can choose the storage for our clients and transports, we can do the same for our connection pools.
```python
import hishel
import httpcore
storage = hishel.RedisStorage()
with httpcore.ConnectionPool() as pool:
with hishel.CacheConnectionPool(pool=pool, storage=storage) as cache_pool:
cache_pool.get("https://example.com/cachable-endpoint")
response = cache_pool.get("https://example.com/cachable-endpoint") # from the cache
```
### Temporarily Disabling the Cache
`Hishel` allows you to temporarily disable the cache for specific requests using the `cache_disabled` extension.
Per RFC9111, the cache can effectively be disabled using the `Cache-Control` headers `no-store` (which requests that the response not be added to the cache),
and `max-age=0` (which demands that any response in the cache must have 0 age - i.e. be a new request). `Hishel` respects this behavior, which can be
used in two ways. First, you can specify the headers directly:
```python
import hishel
import httpx
# With the clients
client = hishel.CacheClient()
client.get(
"https://example.com/cacheable-endpoint",
headers=[("Cache-Control", "no-store"), ("Cache-Control", "max-age=0")]
) # Ignores the cache
# With the transport
cache_transport = hishel.CacheTransport(transport=httpx.HTTPTransport())
client = httpx.Client(transport=cache_transport)
client.get(
"https://example.com/cacheable-endpoint",
headers=[("Cache-Control", "no-store"), ("Cache-Control", "max-age=0")]
) # Ignores the cache
```
Since this can be cumbersome, `Hishel` also provides some "syntactic sugar" to accomplish the same result using `HTTPX` extensions:
```python
import hishel
import httpx
# With the clients
client = hishel.CacheClient()
client.get("https://example.com/cacheable-endpoint", extensions={"cache_disabled": True}) # Ignores the cache
# With the transport
cache_transport = hishel.CacheTransport(transport=httpx.HTTPTransport())
client = httpx.Client(transport=cache_transport)
client.get("https://example.com/cacheable-endpoint", extensions={"cache_disabled": True}) # Ignores the cache
```
Both of these are entirely equivalent to specifying the headers directly.
|