Skip to content

PubSub Publisher #3463

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lukesneeringer merged 44 commits into from
Aug 24, 2017
Merged

PubSub Publisher #3463

Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
f784121
[WIP] Getting started on Pub/Sub.
May 25, 2017
c8a9fd6
stuff
May 25, 2017
9b09b8f
WIP
May 25, 2017
8f0832e
wip
lukesneeringer May 25, 2017
02d7658
WIP
May 30, 2017
3a3ebe9
Merge branch 'pubsub' of github.com:GoogleCloudPlatform/google-cloud-…
May 30, 2017
503d11f
WIP, fixing bugs.
May 30, 2017
3b21b93
WIP
May 30, 2017
1e879a1
wip
May 31, 2017
4bf0552
wip
May 31, 2017
c6dc098
wip
May 31, 2017
13821a7
wip
May 31, 2017
c2d7af8
wip
Jun 1, 2017
4c0bf84
Merge branch 'public-master' into pubsub
Jun 1, 2017
783e9dd
Merge branch 'pubsub' into pubsub-publisher
Jun 2, 2017
7dd719f
Clean up a couple small things.
Jun 2, 2017
c1042ac
A couple more small fixes.
Jun 2, 2017
ccaa865
WIP
Jun 2, 2017
f2ee4d4
Rework based on @jonparrott concurrency ideas.
Jun 2, 2017
12d5546
Refactor the batching implementation.
Jun 2, 2017
e99d959
Remove unrelated files.
Jun 2, 2017
4774f2a
wip
lukesneeringer Jun 3, 2017
1a53c37
Honor size and message count limits.
lukesneeringer Jun 15, 2017
9a6b7cb
Update publisher to be thread-based.
Jul 20, 2017
b0f03a8
Merge branch 'public-master' into pubsub-publisher
Aug 21, 2017
dfe52ef
Merge branch 'pubsub' into pubsub-publisher
Aug 21, 2017
f553fd6
Add better Batch docstring.
Aug 21, 2017
356749a
Improve the max latency thread comments.
Aug 21, 2017
8242c9d
Collapse property docstrings.
Aug 21, 2017
db87dab
More @jonparrott feedback.
Aug 21, 2017
58072b8
Remove the client as a public item in the base batch.
Aug 21, 2017
8f67488
Remove the rejection batch.
Aug 21, 2017
a86d9b7
Lock batch acquisition.
Aug 21, 2017
df18615
Alter exception superclass.
Aug 21, 2017
5f0549b
Inherit from google.api.core.future.Future.
Aug 21, 2017
101d9ca
Move to @jonparrott's Future interface.
Aug 21, 2017
e6e58bb
Move Future off into its own module.
Aug 21, 2017
9fd490c
Add is not None.
Aug 21, 2017
469400e
Feedback.
Aug 22, 2017
31c0c81
Use concurrent.futures.TimeoutError where possible.
Aug 22, 2017
04ac278
Add a lock on commit to ensure no duplication.
Aug 22, 2017
37ee908
Move to an Event.
Aug 22, 2017
12d44be
Make _names private.
Aug 22, 2017
6fe1309
Pubsub subscriber (#3637)
lukesneeringer Aug 24, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions pubsub/google/cloud/gapic/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
__import__('pkg_resources').declare_namespace(__name__)
1 change: 1 addition & 0 deletions pubsub/google/cloud/gapic/pubsub/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
__import__('pkg_resources').declare_namespace(__name__)
0 pubsub/tests/__init__.py → .../google/cloud/gapic/pubsub/v1/__init__.py
View file
Open in desktop
File renamed without changes.
565 changes: 565 additions & 0 deletions pubsub/google/cloud/gapic/pubsub/v1/publisher_client.py
View file
Open in desktop

Large diffs are not rendered by default.

98 changes: 98 additions & 0 deletions pubsub/google/cloud/gapic/pubsub/v1/publisher_client_config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
{
"interfaces": {
"google.pubsub.v1.Publisher": {
"retry_codes": {
"idempotent": [
"DEADLINE_EXCEEDED",
"UNAVAILABLE"
],
"one_plus_delivery": [
"CANCELLED",
"UNKNOWN",
"DEADLINE_EXCEEDED",
"RESOURCE_EXHAUSTED",
"ABORTED",
"INTERNAL",
"UNAVAILABLE"
],
"non_idempotent": [
"UNAVAILABLE"
]
},
"retry_params": {
"default": {
"initial_retry_delay_millis": 100,
"retry_delay_multiplier": 1.3,
"max_retry_delay_millis": 60000,
"initial_rpc_timeout_millis": 60000,
"rpc_timeout_multiplier": 1.0,
"max_rpc_timeout_millis": 60000,
"total_timeout_millis": 600000
},
"messaging": {
"initial_retry_delay_millis": 100,
"retry_delay_multiplier": 1.3,
"max_retry_delay_millis": 60000,
"initial_rpc_timeout_millis": 12000,
"rpc_timeout_multiplier": 1.0,
"max_rpc_timeout_millis": 12000,
"total_timeout_millis": 600000
}
},
"methods": {
"CreateTopic": {
"timeout_millis": 60000,
"retry_codes_name": "idempotent",
"retry_params_name": "default"
},
"Publish": {
"timeout_millis": 60000,
"retry_codes_name": "one_plus_delivery",
"retry_params_name": "messaging",
"bundling": {
"element_count_threshold": 10,
"element_count_limit": 1000,
"request_byte_threshold": 1024,
"request_byte_limit": 10485760,
"delay_threshold_millis": 10
}
},
"GetTopic": {
"timeout_millis": 60000,
"retry_codes_name": "idempotent",
"retry_params_name": "default"
},
"ListTopics": {
"timeout_millis": 60000,
"retry_codes_name": "idempotent",
"retry_params_name": "default"
},
"ListTopicSubscriptions": {
"timeout_millis": 60000,
"retry_codes_name": "idempotent",
"retry_params_name": "default"
},
"DeleteTopic": {
"timeout_millis": 60000,
"retry_codes_name": "idempotent",
"retry_params_name": "default"
},
"SetIamPolicy": {
"timeout_millis": 60000,
"retry_codes_name": "non_idempotent",
"retry_params_name": "default"
},
"GetIamPolicy": {
"timeout_millis": 60000,
"retry_codes_name": "idempotent",
"retry_params_name": "default"
},
"TestIamPermissions": {
"timeout_millis": 60000,
"retry_codes_name": "non_idempotent",
"retry_params_name": "default"
}
}
}
}
}
Loading