3 Matching Annotations
  1. Dec 2023
    1. So what's the deal with asyncio, twisted, gevent, trio and all that stuff?

      asyncio

      asyncio is the modern module for asynchronous network programming provided with the python stdlib since 3.4. In other words, it's the default stuff at your disposal if you want to code something without waiting on the network.

      asyncio replaces the old deprecated asyncore module. It is quite low level, so while you can manually code most network-related things with it, you are still at the level of TCP or UDP. If you want higher-level protocols, like FTP, HTTP or SSH, you have to either code it yourself, or install a third party library or module.

      Because asyncio is the default solution, it has a the biggest ecosystem of 3rd party libs, and pretty much everything async strives to be compatible with it directly, or through compatibility layers like anyio.

      Twisted

      20 years ago, there was no asyncio, there was no async/await, nodejs didn't exist and Python 3 was half a decade away. Yet, it was the .com bubble, everything needed to be connected now. And so was born twisted, the grandfather of all the asynchronous frameworks we have today. Twisted ecosystem grew to include everything, from mail to ssh.

      To this day, twisted is still a robust and versatile tool. But you do pay the price of its age. It doesn't follow PEP8 very well, and the design lean on the heavy size.

      Tornado

      Tornado was developed after Twisted, by FriendFeed, at this weird 2005-2015 web dev period where everything needed to be social web scale. It was like Twisted, but tooted to be faster, and was higher level. Out of the box, the HTTP story is way nicer.

      Today, you are unlikely to use Tornado unless you work at Facebook or contribute to jupyter. After all, if you want to make async web things, the default tool is FastAPI in 2023.

      gevent

      Gevent came about in 2009, the same year as Tornado, but with a fundamentally different design. Instead of attempting to provide an asychronous API, it decided to do black magic. When you use gevent, you call from gevent import monkey; monkey.patch_all() and it changes the underlying mechanism of Python networking, making everything non-blocking.

    2. asyncio, twisted, tornado and gevent have one trick up their sleeve: they can send a message to the network, and while waiting for the response, wake up another part of the program to do some other work. And they can do that with many messages in a row. While waiting for the network, they can let other parts of the program use the CPU core.

      Note that they only can speed up waiting on the network. They will not make two calculations at the same time (can't use several CPU cores like with multiprocessing) and you can't speed up waiting on other types of I/O (like when you use threads to not block on user input or disk writes).

      All in all, they are good for writing things like bots (web crawler, chat bots, network sniffers, etc.) and servers (web servers, proxies, ...). For maximum benefits, it's possible to use them inside other concurrency tools, such as multiprocessing or multithreading. You can perfectly have 4 processes, each of them containing 4 threads (so 16 threads in total), and each thread with their own asyncio loop running.

  2. Sep 2018
    1. 上例中涉及了两个Deferred对象,使用inlineCallbacks装饰器和yield语法,可以用类同步的语法来进行异步编程。

      voltha里面有部分代码使用了Twisted架构