Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bi streaming no debugger stuff #219

Merged
merged 50 commits into from
Jul 31, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
e3955bb
Add initial bi-directional streaming
goodboy May 1, 2021
1411454
Expose `@context` decorator at top level
goodboy May 2, 2021
4846c6d
Cancel scope on stream consumer completion
goodboy May 2, 2021
4240efc
Add basic test set
goodboy May 2, 2021
1f8966b
Support passing `shield` at stream contruction
goodboy May 7, 2021
98133a9
Parametrize with async for style tests
goodboy May 7, 2021
08eb6bd
Fix typing
goodboy May 7, 2021
e311430
Be more pedantic with error handling
goodboy May 10, 2021
e5bc07f
Add dynamic pubsub test using new bidir stream apis
goodboy May 12, 2021
6559fb7
Expose msg stream types at top level
goodboy May 12, 2021
a2e2f7e
Only send stop msg if not received from far end
goodboy May 12, 2021
9a4244b
Support no arg to `Context.started()` like trio
goodboy May 25, 2021
5b8b7d3
Add error case
goodboy Jun 10, 2021
39b9896
Only close recv chan if we get a ref
goodboy Jun 10, 2021
3999849
Add a multi-task streaming test
goodboy Jun 10, 2021
eb3662f
Add a specially handled `ContextCancelled` error
goodboy Jun 13, 2021
409f7f0
Expose streaming components at top level
goodboy Jun 13, 2021
73302d9
Specially raise a `ContextCancelled` for a task-context rpc
goodboy Jun 13, 2021
348148f
Explicitly formalize context/streaming teardown
goodboy Jun 13, 2021
1a69727
Fix exception typing
goodboy Jun 14, 2021
54916be
Adjustments for non-frozen context dataclass change
goodboy Jun 14, 2021
196dea8
Drop trailing comma
goodboy Jun 14, 2021
7c5fd8c
Add detailed ``@tractor.context`` cancellation/termination tests
goodboy Jun 14, 2021
349d82d
Speedup the dynamic pubsub test
goodboy Jun 14, 2021
8eb889a
Modernize streaming tests
goodboy Jun 14, 2021
3d63340
Don't clobber msg loop mem chan on rx stream close
goodboy Jun 14, 2021
1703171
Set stream "end of channel" after shielded check!
goodboy Jun 14, 2021
af701c1
Consider relaying context error via raised-in-scope-nursery task
goodboy Jun 24, 2021
3423ea4
Add temp warning msg for context cancel call
goodboy Jun 27, 2021
c2484e8
First try: pack cancelled tracebacks and ship to caller
goodboy Jun 27, 2021
91640fa
Always shield cancel the caller on cancel-causing-errors, add teardow…
goodboy Jun 28, 2021
c6cdaf9
De-densify some code
goodboy Jun 30, 2021
b21e2a6
Add pre-stream open error conditions
goodboy Jun 30, 2021
0623de0
Expect context cancelled when we cancel
goodboy Jun 30, 2021
a134bc4
Avoid mutate during interate error
goodboy Jun 30, 2021
ef725c5
Always hard kill sub-procs on teardown
goodboy Jun 26, 2021
b1cd7fd
Don't shield on root cancel it can causes hangs
goodboy May 30, 2021
98bbf8e
Move join event trigger to direct exit path
goodboy Jul 5, 2021
12f9875
Don't enter debug on closed resource errors
goodboy Jul 5, 2021
7f86d63
Drop trip kwarg
goodboy Jul 5, 2021
9ddb654
Avoid mutate on iterate race
goodboy Jun 10, 2021
2513c65
Go back to only logging crashes if no pdb gets engaged
goodboy Jun 29, 2021
929b6dc
Skip debugger tests on windows at module level
goodboy Jul 6, 2021
31590e8
Flip "trace" level to "transport" level logging
goodboy Jun 30, 2021
8c927d7
Change trace to transport level
goodboy Jun 30, 2021
fde52d2
Mypy fixes
goodboy Jul 8, 2021
25779d4
Define explicit adapter level methods for mypy
goodboy Jul 8, 2021
443ebea
Use "pdb" level logging in debug mode
goodboy Jul 8, 2021
69bbf6a
Install test deps and py3.9 for type check job
goodboy Jul 8, 2021
240f591
Add 2-way streaming example to readme and scripts
goodboy Jul 31, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,19 @@ jobs:
mypy:
name: 'MyPy'
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v2

- name: Setup python
uses: actions/setup-python@v2
with:
python-version: '3.8'
python-version: '3.9'

- name: Install dependencies
run: pip install -U . --upgrade-strategy eager
run: pip install -U . --upgrade-strategy eager -r requirements-test.txt

- name: Run MyPy check
run: mypy tractor/ --ignore-missing-imports

Expand Down
95 changes: 94 additions & 1 deletion docs/README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,8 @@ Zombie safe: self-destruct a process tree
print('This process tree will self-destruct in 1 sec...')
await trio.sleep(1)

# you could have done this yourself
# raise an error in root actor/process and trigger
# reaping of all minions
raise Exception('Self Destructed')


Expand Down Expand Up @@ -197,6 +198,98 @@ And, yes, there's a built-in crash handling mode B)
We're hoping to add a respawn-from-repl system soon!


SC compatible bi-directional streaming
--------------------------------------
Yes, you saw it here first; we provide 2-way streams
with reliable, transitive setup/teardown semantics.

Our nascent api is remniscent of ``trio.Nursery.start()``
style invocation:

.. code:: python

import trio
import tractor


@tractor.context
async def simple_rpc(

ctx: tractor.Context,
data: int,

) -> None:
'''Test a small ping-pong 2-way streaming server.

'''
# signal to parent that we're up much like
# ``trio_typing.TaskStatus.started()``
await ctx.started(data + 1)

async with ctx.open_stream() as stream:

count = 0
async for msg in stream:

assert msg == 'ping'
await stream.send('pong')
count += 1

else:
assert count == 10


async def main() -> None:

async with tractor.open_nursery() as n:

portal = await n.start_actor(
'rpc_server',
enable_modules=[__name__],
)

# XXX: this syntax requires py3.9
async with (

portal.open_context(
simple_rpc,
data=10,
) as (ctx, sent),

ctx.open_stream() as stream,
):

assert sent == 11

count = 0
# receive msgs using async for style
await stream.send('ping')

async for msg in stream:
assert msg == 'pong'
await stream.send('ping')
count += 1

if count >= 9:
break


# explicitly teardown the daemon-actor
await portal.cancel_actor()


if __name__ == '__main__':
trio.run(main)


See original proposal and discussion in `#53`_ as well
as follow up improvements in `#223`_ that we'd love to
hear your thoughts on!

.. _#53: https://github.com/goodboy/tractor/issues/53
.. _#223: https://github.com/goodboy/tractor/issues/223


Worker poolz are easy peasy
---------------------------
The initial ask from most new users is *"how do I make a worker
Expand Down
72 changes: 72 additions & 0 deletions examples/rpc_bidir_streaming.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
import trio
import tractor


@tractor.context
async def simple_rpc(

ctx: tractor.Context,
data: int,

) -> None:
'''Test a small ping-pong 2-way streaming server.

'''
# signal to parent that we're up much like
# ``trio_typing.TaskStatus.started()``
await ctx.started(data + 1)

async with ctx.open_stream() as stream:

count = 0
async for msg in stream:

assert msg == 'ping'
await stream.send('pong')
count += 1

else:
assert count == 10


async def main() -> None:

async with tractor.open_nursery() as n:

portal = await n.start_actor(
'rpc_server',
enable_modules=[__name__],
)

# XXX: syntax requires py3.9
async with (

portal.open_context(
simple_rpc, # taken from pytest parameterization
data=10,

) as (ctx, sent),

ctx.open_stream() as stream,
):

assert sent == 11

count = 0
# receive msgs using async for style
await stream.send('ping')

async for msg in stream:
assert msg == 'pong'
await stream.send('ping')
count += 1

if count >= 9:
break

# explicitly teardown the daemon-actor
await portal.cancel_actor()


if __name__ == '__main__':
trio.run(main)
Loading