1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
|
<html><body>
<style>
body, h1, h2, h3, div, span, p, pre, a {
margin: 0;
padding: 0;
border: 0;
font-weight: inherit;
font-style: inherit;
font-size: 100%;
font-family: inherit;
vertical-align: baseline;
}
body {
font-size: 13px;
padding: 1em;
}
h1 {
font-size: 26px;
margin-bottom: 1em;
}
h2 {
font-size: 24px;
margin-bottom: 1em;
}
h3 {
font-size: 20px;
margin-bottom: 1em;
margin-top: 1em;
}
pre, code {
line-height: 1.5;
font-family: Monaco, 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Lucida Console', monospace;
}
pre {
margin-top: 0.5em;
}
h1, h2, h3, p {
font-family: Arial, sans serif;
}
h1, h2, h3 {
border-bottom: solid #CCC 1px;
}
.toc_element {
margin-top: 0.5em;
}
.firstline {
margin-left: 2 em;
}
.method {
margin-top: 1em;
border: solid 1px #CCC;
padding: 1em;
background: #EEE;
}
.details {
font-weight: bold;
font-size: 14px;
}
</style>
<h1><a href="dataflow_v1b3.html">Dataflow API</a> . <a href="dataflow_v1b3.projects.html">projects</a></h1>
<h2>Instance Methods</h2>
<p class="toc_element">
<code><a href="dataflow_v1b3.projects.jobs.html">jobs()</a></code>
</p>
<p class="firstline">Returns the jobs Resource.</p>
<p class="toc_element">
<code><a href="dataflow_v1b3.projects.locations.html">locations()</a></code>
</p>
<p class="firstline">Returns the locations Resource.</p>
<p class="toc_element">
<code><a href="dataflow_v1b3.projects.snapshots.html">snapshots()</a></code>
</p>
<p class="firstline">Returns the snapshots Resource.</p>
<p class="toc_element">
<code><a href="dataflow_v1b3.projects.templates.html">templates()</a></code>
</p>
<p class="firstline">Returns the templates Resource.</p>
<p class="toc_element">
<code><a href="#deleteSnapshots">deleteSnapshots(projectId, location=None, snapshotId=None, x__xgafv=None)</a></code></p>
<p class="firstline">Deletes a snapshot.</p>
<p class="toc_element">
<code><a href="#workerMessages">workerMessages(projectId, body, x__xgafv=None)</a></code></p>
<p class="firstline">Send a worker_message to the service.</p>
<h3>Method Details</h3>
<div class="method">
<code class="details" id="deleteSnapshots">deleteSnapshots(projectId, location=None, snapshotId=None, x__xgafv=None)</code>
<pre>Deletes a snapshot.
Args:
projectId: string, The ID of the Cloud Platform project that the snapshot belongs to. (required)
location: string, The location that contains this snapshot.
snapshotId: string, The ID of the snapshot.
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{ # Response from deleting a snapshot.
}</pre>
</div>
<div class="method">
<code class="details" id="workerMessages">workerMessages(projectId, body, x__xgafv=None)</code>
<pre>Send a worker_message to the service.
Args:
projectId: string, The project to send the WorkerMessages to. (required)
body: object, The request body. (required)
The object takes the form of:
{ # A request for sending worker messages to the service.
"workerMessages": [ # The WorkerMessages to send.
{ # WorkerMessage provides information to the backend about a worker.
"workerShutdownNotice": { # Shutdown notification from workers. This is to be sent by the shutdown # Shutdown notice by workers.
# script of the worker VM so that the backend knows that the VM is being
# shut down.
"reason": "A String", # The reason for the worker shutdown.
# Current possible values are:
# "UNKNOWN": shutdown reason is unknown.
# "PREEMPTION": shutdown reason is preemption.
# Other possible reasons may be added in the future.
},
"workerMetrics": { # Worker metrics exported from workers. This contains resource utilization # Resource metrics reported by workers.
# metrics accumulated from a variety of sources. For more information, see
# go/df-resource-signals.
"cpuTime": [ # CPU utilization samples.
{ # Modeled after information exposed by /proc/stat.
"totalMs": "A String", # Total active CPU time across all cores (ie., non-idle) in milliseconds
# since start-up.
"rate": 3.14, # Average CPU utilization rate (% non-idle cpu / second) since previous
# sample.
"timestamp": "A String", # Timestamp of the measurement.
},
],
},
"workerMessageCode": { # A message code is used to report status and error messages to the service. # A worker message code.
# The message codes are intended to be machine readable. The service will
# take care of translating these into user understandable messages if
# necessary.
#
# Example use cases:
# 1. Worker processes reporting successful startup.
# 2. Worker processes reporting specific errors (e.g. package staging
# failure).
"code": "A String", # The code is a string intended for consumption by a machine that identifies
# the type of message being sent.
# Examples:
# 1. "HARNESS_STARTED" might be used to indicate the worker harness has
# started.
# 2. "GCS_DOWNLOAD_ERROR" might be used to indicate an error downloading
# a GCS file as part of the boot process of one of the worker containers.
#
# This is a string and not an enum to make it easy to add new codes without
# waiting for an API change.
"parameters": { # Parameters contains specific information about the code.
#
# This is a struct to allow parameters of different types.
#
# Examples:
# 1. For a "HARNESS_STARTED" message parameters might provide the name
# of the worker and additional data like timing information.
# 2. For a "GCS_DOWNLOAD_ERROR" parameters might contain fields listing
# the GCS objects being downloaded and fields containing errors.
#
# In general complex data structures should be avoided. If a worker
# needs to send a specific and complicated data structure then please
# consider defining a new proto and adding it to the data oneof in
# WorkerMessageResponse.
#
# Conventions:
# Parameters should only be used for information that isn't typically passed
# as a label.
# hostname and other worker identifiers should almost always be passed
# as labels since they will be included on most messages.
"a_key": "", # Properties of the object.
},
},
"labels": { # Labels are used to group WorkerMessages.
# For example, a worker_message about a particular container
# might have the labels:
# { "JOB_ID": "2015-04-22",
# "WORKER_ID": "wordcount-vm-2015…"
# "CONTAINER_TYPE": "worker",
# "CONTAINER_ID": "ac1234def"}
# Label tags typically correspond to Label enum values. However, for ease
# of development other strings can be used as tags. LABEL_UNSPECIFIED should
# not be used here.
"a_key": "A String",
},
"workerLifecycleEvent": { # A report of an event in a worker's lifecycle. # Record of worker lifecycle events.
# The proto contains one event, because the worker is expected to
# asynchronously send each message immediately after the event.
# Due to this asynchrony, messages may arrive out of order (or missing), and it
# is up to the consumer to interpret.
# The timestamp of the event is in the enclosing WorkerMessage proto.
"event": "A String", # The event being reported.
"containerStartTime": "A String", # The start time of this container. All events will report this so that
# events can be grouped together across container/VM restarts.
"metadata": { # Other stats that can accompany an event. E.g.
# { "downloaded_bytes" : "123456" }
"a_key": "A String",
},
},
"time": "A String", # The timestamp of the worker_message.
"workerHealthReport": { # WorkerHealthReport contains information about the health of a worker. # The health of a worker.
#
# The VM should be identified by the labels attached to the WorkerMessage that
# this health ping belongs to.
"vmIsBroken": True or False, # Whether the VM is in a permanently broken state.
# Broken VMs should be abandoned or deleted ASAP to avoid assigning or
# completing any work.
"vmStartupTime": "A String", # The time the VM was booted.
"vmIsHealthy": True or False, # Whether the VM is currently healthy.
"msg": "A String", # A message describing any unusual health reports.
"reportInterval": "A String", # The interval at which the worker is sending health reports.
# The default value of 0 should be interpreted as the field is not being
# explicitly set by the worker.
"pods": [ # The pods running on the worker. See:
# http://kubernetes.io/v1.1/docs/api-reference/v1/definitions.html#_v1_pod
#
# This field is used by the worker to send the status of the indvidual
# containers running on each worker.
{
"a_key": "", # Properties of the object.
},
],
},
},
],
"location": "A String", # The [regional endpoint]
# (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that
# contains the job.
}
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{ # The response to the worker messages.
"workerMessageResponses": [ # The servers response to the worker messages.
{ # A worker_message response allows the server to pass information to the
# sender.
"workerHealthReportResponse": { # WorkerHealthReportResponse contains information returned to the worker # The service's response to a worker's health report.
# in response to a health ping.
"reportInterval": "A String", # A positive value indicates the worker should change its reporting interval
# to the specified value.
#
# The default value of zero means no change in report rate is requested by
# the server.
},
"workerMetricsResponse": { # Service-side response to WorkerMessage reporting resource utilization. # Service's response to reporting worker metrics (currently empty).
},
"workerShutdownNoticeResponse": { # Service-side response to WorkerMessage issuing shutdown notice. # Service's response to shutdown notice (currently empty).
},
},
],
}</pre>
</div>
</body></html>
|
