File: error_handling.md

package info (click to toggle)
ruby-bunny 2.23.0-1
  • links: PTS, VCS
  • area: main
  • in suites: forky, sid, trixie
  • size: 1,644 kB
  • sloc: ruby: 10,256; sh: 70; makefile: 8
file content (305 lines) | stat: -rw-r--r-- 10,969 bytes parent folder | download | duplicates (2)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
---
title: "Error Handling and Recovery"
layout: article
---

## About this guide

Development of a robust application, be it message publisher or
message consumer, involves dealing with multiple kinds of failures:
protocol exceptions, network failures, broker failures and so
on. Correct error handling and recovery is not easy. This guide
explains how the library helps you in dealing with issues like

 * Client exceptions
 * Initial connection failures
 * Network connection failures
 * AMQP 0.9.1 connection-level exceptions
 * AMQP 0.9.1 channel-level exceptions
 * Broker failure
 * TLS (SSL) related issues

as well as

 * How does the automatic recovery mode in Bunny 0.9+ work

This work is licensed under a <a rel="license"
href="http://creativecommons.org/licenses/by/3.0/">Creative Commons
Attribution 3.0 Unported License</a> (including images and
stylesheets). The source is available [on
GitHub](https://github.com/ruby-amqp/rubybunny.info).


## What version of Bunny does this guide cover?

This guide covers Bunny 2.11.0 and later versions.


## Client Exceptions

Here is the break-down of exceptions that can be raised by Bunny:

    StandardError
      Bunny::Exception
        Bunny::ChannelAlreadyClosed

        Bunny::ChannelLevelException
          Bunny::AccessRefused
          Bunny::ForcedChannelCloseError
          Bunny::NotFound
          Bunny::PreconditionFailed
          Bunny::ResourceLocked

        Bunny::ConnectionClosedError

        Bunny::ConnectionLevelException
          Bunny::ChannelError
          Bunny::CommandInvalid
          Bunny::ConnectionForced
          Bunny::ForcedConnectionCloseError
          Bunny::FrameError
          Bunny::InternalError
          Bunny::ResourceError
          Bunny::UnexpectedFrame

        Bunny::InconsistentDataError
          Bunny::BadLengthError
          Bunny::NoFinalOctetError

        Bunny::NetworkFailure
        Bunny::NotAllowedError

        Bunny::PossibleAuthenticationFailureError
          Bunny::AuthenticationFailureError

        Bunny::ShutdownSignal
        Bunny::TCPConnectionFailed

    Timeout::Error
      Bunny::ClientTimeout
      Bunny::ConnectionTimeout

The rest of the document describes the most common ones.
See [Bunny exception definitions](https://raw.githubusercontent.com/ruby-amqp/bunny/master/lib/bunny/exceptions.rb) for more details.


## Initial RabbitMQ Connection Failures

When applications connect to the broker, they need to handle
connection failures. Networks are not 100% reliable, even with modern
system configuration tools like Chef or Puppet misconfigurations
happen and the broker might also be down. Error detection should
happen as early as possible. To handle TCP
connection failure, catch the `Bunny::TCPConnectionFailure` exception:

``` ruby
begin
  conn = Bunny.new("amqp://guest:guest@aksjhdkajshdkj.example82737.com")
  conn.start
rescue Bunny::TCPConnectionFailed => e
  puts "Connection to aksjhdkajshdkj.example82737.com failed"
end
```

`Bunny::Session#start` will raise `Bunny::TCPConnectionFailed` if a
connection fails. Code that catches it can write to a log about the
issue or use retry to execute the begin block one more time. Because
initial connection failures are due to misconfiguration or network
outage, reconnection to the same endpoint (hostname, port, vhost
combination) may result in the same issue over and over.


## Authentication Failures

Another reason why a connection may fail is authentication
failure. Handling authentication failure is very similar to handling
initial TCP connection failure:

``` ruby
begin
  conn = Bunny.new("amqp://guest8we78w7e8:guest2378278@127.0.0.1")
  conn.start
rescue Bunny::PossibleAuthenticationFailureError => e
  puts "Could not authenticate as #{conn.username}"
end
```

In case you are wondering why the exception name has "possible" in it:
[AMQP 0.9.1 spec](http://www.rabbitmq.com/resources/specs/amqp0-9-1.pdf) requires broker
implementations to simply close TCP connection without sending any
more data when an exception (such as authentication failure) occurs
before AMQP connection is open. In practice, however, when broker
closes TCP connection between successful TCP connection and before
AMQP connection is open, it means that authentication has failed.

RabbitMQ 3.2 introduces [authentication failure notifications](http://www.rabbitmq.com/auth-notification.html)
which Bunny supports. When connecting to RabbitMQ 3.2 or later, Bunny will
raise `Bunny::AuthenticationFailureError` when it receives a proper
authentication failure notification.


## Network Connection Failures

Detecting network connections is nearly useless if an application
cannot recover from them. Recovery is the hard part in "error handling
and recovery". Fortunately, the recovery process for many applications
follows a common scheme that Bunny can perform automatically for
you.

When Bunny detects TCP connection failure, it will try to reconnect
every 5 seconds. Currently there is no limit on the number of reconnection
attempts.

To disable automatic connection recovery, pass `:automatic_recovery => false`
to `Bunny.new`.

### Server-Initiated `connection.close`

Server-initiated `connection.close` (issued due to an unrecoverable client
issue or when a connection is forced to close via RabbitMQ management UI/HTTP API
or when a server is shutting down)will result in an exception on the thread
`Bunny::Session` was instantiated.

Bunny can be instructed from such exceptions (see Automatic Recovery below).


### Automatic Recovery

Many applications use the same recovery strategy that consists of the following steps:

 * Re-open channels
 * For each channel, re-declare exchanges (except for predefined ones)
 * For each channel, re-declare queues
 * For each queue, recover all bindings
 * For each queue, recover all consumers

Bunny provides a feature known as "automatic recovery" that performs these steps
after connection recovery, while taking care of some of the more tricky details
such as recovery of server-named queues with consumers.

Currently the topology recovery strategy is not configurable.

When automatic recovery is disabled, Bunny will raise
exceptions on the thread `Bunny::Session` was instantiated on.

Bunny will recover from server-sent `connection.close`, if you don't want it to do
so then pass `recover_from_connection_close: false` to `Bunny.new`. 


## Channel-level Exceptions

Channel-level exceptions are more common than connection-level ones and often indicate
issues applications can recover from (such as consuming from or trying to delete
a queue that does not exist).

With Bunny, channel-level exceptions are raised as Ruby exceptions, for example,
`Bunny::NotFound`, that provide access to the underlying `channel.close` method
information:

``` ruby
begin
  ch.queue_delete("queue_that_should_not_exist#{rand}")
rescue Bunny::NotFound => e
  puts "Channel-level exception! Code: #{e.channel_close.reply_code}, message: #{e.channel_close.reply_text}"
end
```

``` ruby
begin
  ch2 = conn.create_channel
  q   = "bunny.examples.recovery.q#{rand}"

  ch2.queue_declare(q, :durable => false)
  ch2.queue_declare(q, :durable => true)
rescue Bunny::PreconditionFailed => e
  puts "Channel-level exception! Code: #{e.channel_close.reply_code}, message: #{e.channel_close.reply_text}"
ensure
  conn.create_channel.queue_delete(q)
end
```


### Common channel-level exceptions and what they mean

A few channel-level exceptions are common and deserve more attention.

#### 406 Precondition Failed

<dl>
  <dt>Description</dt>
  <dd>The client requested a method that was not allowed because some precondition failed.</dd>
  <dt>What might cause it</dt>
  <dd>
    <ul>
      <li>AMQP entity (a queue or exchange) was re-declared with attributes different from original declaration. Maybe two applications or pieces of code declare the same entity with different attributes. Note that different RabbitMQ client libraries historically use slightly different defaults for entities and this may cause attribute mismatches.</li>
      <li>`Bunny::Channel#tx_commit` or `Bunny::Channel#tx_rollback` might be run on a channel that wasn't previously made transactional with `Bunny::Channel#tx_select`</li>
    </ul>
  </dd>
  <dt>Example RabbitMQ error message</dt>
  <dd>
    <ul>
      <li>PRECONDITION_FAILED - parameters for queue 'bunny.examples.channel_exception' in vhost '/' not equivalent</li>
      <li>PRECONDITION_FAILED - channel is not transactional</li>
    </ul>
  </dd>
</dl>

#### 405 Resource Locked

<dl>
  <dt>Description</dt>
  <dd>The client attempted to work with a server entity to which it has no access because another client is working with it.</dd>
  <dt>What might cause it</dt>
  <dd>
    <ul>
      <li>Multiple applications (or different pieces of code/threads/processes/routines within a single application) might try to declare queues with the same name as exclusive.</li>
      <li>Multiple consumer across multiple or single app might be registered as exclusive for the same queue.</li>
    </ul>
  </dd>
  <dt>Example RabbitMQ error message</dt>
  <dd>RESOURCE_LOCKED - cannot obtain exclusive access to locked queue 'bunny.examples.queue' in vhost '/'</dd>
</dl>

#### 404 Not Found

<dl>
  <dt>Description</dt>
  <dd>The client attempted to use (publish to, delete, etc) an entity (exchange, queue) that does not exist.</dd>
  <dt>What might cause it</dt>
  <dd>Application miscalculates queue or exchange name or tries to use an entity that was deleted earlier</dd>
  <dt>Example RabbitMQ error message</dt>
  <dd>NOT_FOUND - no queue 'queue_that_should_not_exist0.6798199937619038' in vhost '/'</dd>
</dl>

#### 403 Access Refused

<dl>
  <dt>Description</dt>
  <dd>The client attempted to work with a server entity to which it has no access due to security settings.</dd>
  <dt>What might cause it</dt>
  <dd>Application tries to access a queue or exchange it has no permissions for (or right kind of permissions, for example, write permissions)</dd>
  <dt>Example RabbitMQ error message</dt>
  <dd>ACCESS_REFUSED - access to queue 'bunny.examples.channel_exception' in vhost 'bunny_testbed' refused for user 'bunny_reader'</dd>
</dl>




## What to Read Next

The documentation is organized as [a number of
guides](/articles/guides.html), covering various topics.

We recommend that you read the following guides first, if possible, in this order:

 * [Troubleshooting](/articles/troubleshooting.html)
 * [Using TLS (SSL) Connections](/articles/tls.html)


## Tell Us What You Think!

Please take a moment to tell us what you think about this guide [on Twitter](http://twitter.com/rubyamqp) or the [Bunny mailing list](https://groups.google.com/forum/#!forum/ruby-amqp)

Let us know what was unclear or what has not been covered. Maybe you
do not like the guide style or grammar or discover spelling
mistakes. Reader feedback is key to making the documentation better.