- Mar 2018
-
docs.sqlalchemy.org docs.sqlalchemy.org
-
max_overflow
This defaults to 10 (assuming we increase our pool_size to be 10) which means we will have max 20 connections.
-
pool_size¶
The default is 5 in our env -but we typically run with at least 10 connections per instance so this prob means we should increase this number in our config to 10.
-
from sqlalchemy import create_engine, exc e = create_engine(...) c = e.connect()
It seems like this is the way we should be using the db engine. Create it once, then create 'sessions' / 'connection' off of it throughout the code. We are doing this by using sessions that underneath grab a connection from the pool. See https://github.com/hypothesis/h/blob/master/h/streamer/streamer.py.
-
All SQLAlchemy pool implementations have in common that none of them “pre create” connections - all implementations wait until first use before creating a connection. At that point, if no additional concurrent checkout requests for more connections are made, no additional connections are created. This is why it’s perfectly fine for create_engine() to default to using a QueuePool of size five without regard to whether or not the application really needs five connections queued up - the pool would only grow to that size if the application actually used five connections concurrently, in which case the usage of a small pool is an entirely appropriate default behavior.
The default connection pool is a "small pool"-as other data seems to indicate thus further pointing to increasing our pool size would be a good place to start. Also note the default pool doesn't start out with all those connections it ramps up and discards all connections above pool_size.
-
Particularly for server-side web applications, a connection pool is the standard way to maintain a “pool” of active database connections in memory which are reused across requests.
This is something that appears to be not currently happening as our 'default state' number of connections is 50 in total. Rather than taking the thrash as traffic ramps we should maintain more connections in the queue by default. This will avoid cpu thrash as creating connections is an expensive cpu thrashing operation.
-
-
github.com github.com
-
session = session_factory(settings)
This is where we create the db engine/connection pool and a session associated with it.Note at the end we close the session-but not the engine or the pool. The connection created by the original session will be re-used each iteration. See http://docs.sqlalchemy.org/en/latest/core/pooling.html and http://docs.sqlalchemy.org/en/latest/orm/session_basics.html#getting-a-session for details.
-
-
docs.sqlalchemy.org docs.sqlalchemy.org
-
Session = sessionmaker(bind=some_engine)
A session is bound to the engine and therefore it's resources. When a session is created it grabs a connection from the existing pool of resources bound to the engine.
-