Connection pool base class for managing Active Record database connections.
Introduction
A connection pool synchronizes thread access to a limited number of database connections. The basic idea is that each thread checks out a database connection from the pool, uses that connection, and checks the connection back in. ConnectionPool is completely thread-safe, and will ensure that a connection cannot be used by two threads at the same time, as long as ConnectionPool's contract is correctly followed. It will also handle cases in which there are more threads than connections: if all connections have been checked out, and a thread tries to checkout a connection anyway, then ConnectionPool will wait until some other thread has checked in a connection.
Obtaining (checking out) a connection
Connections can be obtained and used from a connection pool in several ways:
-
Simply use ActiveRecord::Base.connection as with Active Record 2.1 and earlier (pre-connection-pooling). Eventually, when you're done with the connection(s) and wish it to be returned to the pool, you call ActiveRecord::Base.clear_active_connections!. This will be the default behavior for Active Record when used in conjunction with Action Pack's request handling cycle.
-
Manually check out a connection from the pool with ActiveRecord::Base.connection_pool.checkout. You are responsible for returning this connection to the pool when finished by calling ActiveRecord::Base.connection_pool.checkin(connection).
-
Use ActiveRecord::Base.connection_pool.with_connection(&block), which obtains a connection, yields it as the sole argument to the block, and returns it to the pool after the block completes.
Connections in the pool are actually AbstractAdapter objects (or objects compatible with AbstractAdapter's interface).
Options
There are several connection-pooling-related options that you can add to your database connection configuration:
-
pool
: number indicating size of connection pool (default 5) -
checkout_timeout
: number of seconds to block and wait for a connection before giving up and raising a timeout error (default 5 seconds). -
reaping_frequency
: frequency in seconds to periodically run the Reaper, which attempts to find and recover connections from dead threads, which can occur if a programmer forgets to close a connection at the end of a thread or a thread dies unexpectedly. Regardless of this setting, the Reaper will be invoked before every blocking wait. (Defaultnil
, which means don't schedule the Reaper).
- CLASS ActiveRecord::ConnectionAdapters::ConnectionPool::Queue
- CLASS ActiveRecord::ConnectionAdapters::ConnectionPool::Reaper
- A
- C
- D
- L
- N
- R
- S
- W
[RW] | automatic_reconnect | |
[RW] | checkout_timeout | |
[R] | connections | |
[R] | reaper | |
[RW] | schema_cache | |
[R] | size | |
[R] | spec |
Creates a new ConnectionPool object.
spec
is a ConnectionSpecification object which describes
database connection information (e.g. adapter, host name, username,
password, etc), as well as the maximum size for this ConnectionPool.
The default ConnectionPool maximum size is 5.
# File activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb, line 321 def initialize(spec) super() @spec = spec @checkout_timeout = (spec.config[:checkout_timeout] && spec.config[:checkout_timeout].to_f) || 5 @reaper = Reaper.new(self, (spec.config[:reaping_frequency] && spec.config[:reaping_frequency].to_f)) @reaper.run # default max pool size to 5 @size = (spec.config[:pool] && spec.config[:pool].to_i) || 5 # This variable tracks the cache of threads mapped to reserved connections, with the # sole purpose of speeding up the +connection+ method. It is not the authoritative # registry of which thread owns which connection. Connection ownership is tracked by # the +connection.owner+ attr on each +connection+ instance. # The invariant works like this: if there is mapping of <tt>thread => conn</tt>, # then that +thread+ does indeed own that +conn+. However, an absence of a such # mapping does not mean that the +thread+ doesn't own the said connection. In # that case +conn.owner+ attr should be consulted. # Access and modification of +@thread_cached_conns+ does not require # synchronization. @thread_cached_conns = Concurrent::Map.new(initial_capacity: @size) @connections = [] @automatic_reconnect = true # Connection pool allows for concurrent (outside the main +synchronize+ section) # establishment of new connections. This variable tracks the number of threads # currently in the process of independently establishing connections to the DB. @now_connecting = 0 @threads_blocking_new_connections = 0 @available = ConnectionLeasingQueue.new self @lock_thread = false end
Returns true if there is an open connection being used for the current thread.
This method only works for connections that have been obtained through connection or with_connection methods. Connections obtained through checkout will not be detected by active_connection?
Check-in a database connection back into the pool, indicating that you no longer need this connection.
conn
: an AbstractAdapter
object, which was obtained by earlier by calling checkout on this pool.
Check-out a database connection from the pool, indicating that you want to use it. You should call checkin when you no longer need this.
This is done by either returning and leasing existing connection, or by creating a new connection and leasing it.
If all connections are leased and the pool is at capacity (meaning the number of currently leased connections is greater than or equal to the size limit set), an ActiveRecord::ConnectionTimeoutError exception will be raised.
Returns: an AbstractAdapter object.
Raises:
-
ActiveRecord::ConnectionTimeoutError no connection can be obtained from the pool.
Clears the cache which maps classes and re-connects connections that require reloading.
Raises:
-
ActiveRecord::ExclusiveConnectionTimeoutError if unable to gain ownership of all connections in the pool within a timeout interval (default duration is
spec.config[:checkout_timeout] * 2
seconds).
# File activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb, line 457 def clear_reloadable_connections(raise_on_acquisition_timeout = true) with_exclusively_acquired_all_connections(raise_on_acquisition_timeout) do synchronize do @connections.each do |conn| if conn.in_use? conn.steal! checkin conn end conn.disconnect! if conn.requires_reloading? end @connections.delete_if(&:requires_reloading?) @available.clear end end end
Clears the cache which maps classes and re-connects connections that require reloading.
The pool first tries to gain ownership of all connections. If unable to do
so within a timeout interval (default duration is
spec.config[:checkout_timeout] * 2
seconds), then the pool
forcefully clears the cache and reloads connections without any regard for
other connection owning threads.
Returns true if a connection has already been opened.
Retrieve the connection associated with the current thread, or call checkout to obtain one if necessary.
connection can be called any number of times; the connection is held in a cache keyed by a thread.
Disconnects all connections in the pool, and clears the pool.
Raises:
-
ActiveRecord::ExclusiveConnectionTimeoutError if unable to gain ownership of all connections in the pool within a timeout interval (default duration is
spec.config[:checkout_timeout] * 2
seconds).
# File activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb, line 424 def disconnect(raise_on_acquisition_timeout = true) with_exclusively_acquired_all_connections(raise_on_acquisition_timeout) do synchronize do @connections.each do |conn| if conn.in_use? conn.steal! checkin conn end conn.disconnect! end @connections = [] @available.clear end end end
Disconnects all connections in the pool, and clears the pool.
The pool first tries to gain ownership of all connections. If unable to do
so within a timeout interval (default duration is
spec.config[:checkout_timeout] * 2
seconds), then the pool is
forcefully disconnected without any regard for other connection owning
threads.
Recover lost connections for the pool. A lost connection can occur if a programmer forgets to checkin a connection at the end of a thread or a thread dies unexpectedly.
# File activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb, line 556 def reap stale_connections = synchronize do @connections.select do |conn| conn.in_use? && !conn.owner.alive? end.each do |conn| conn.steal! end end stale_connections.each do |conn| if conn.active? conn.reset! checkin conn else remove conn end end end
Signal that the thread is finished with the current connection. release_connection releases the connection-thread association and returns the connection to the pool.
This method only works for connections that have been obtained through connection or with_connection methods, connections obtained through checkout will not be automatically released.
Remove a connection from the connection pool. The connection will remain open and active but will no longer be managed by this pool.
# File activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb, line 524 def remove(conn) needs_new_connection = false synchronize do remove_connection_from_thread_cache conn @connections.delete conn @available.delete conn # @available.any_waiting? => true means that prior to removing this # conn, the pool was at its max size (@connections.size == @size). # This would mean that any threads stuck waiting in the queue wouldn't # know they could checkout_new_connection, so let's do it for them. # Because condition-wait loop is encapsulated in the Queue class # (that in turn is oblivious to ConnectionPool implementation), threads # that are "stuck" there are helpless. They have no way of creating # new connections and are completely reliant on us feeding available # connections into the Queue. needs_new_connection = @available.any_waiting? end # This is intentionally done outside of the synchronized section as we # would like not to hold the main mutex while checking out new connections. # Thus there is some chance that needs_new_connection information is now # stale, we can live with that (bulk_make_new_connections will make # sure not to exceed the pool's @size limit). bulk_make_new_connections(1) if needs_new_connection end
Return connection pool's usage statistic Example:
ActiveRecord::Base.connection_pool.stat # => { size: 15, connections: 1, busy: 1, dead: 0, idle: 0, waiting: 0, checkout_timeout: 5 }
# File activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb, line 583 def stat synchronize do { size: size, connections: @connections.size, busy: @connections.count { |c| c.in_use? && c.owner.alive? }, dead: @connections.count { |c| c.in_use? && !c.owner.alive? }, idle: @connections.count { |c| !c.in_use? }, waiting: num_waiting_in_queue, checkout_timeout: checkout_timeout } end end
If a connection obtained through connection or with_connection methods already exists yield it to the block. If no such connection exists checkout a connection, yield it to the block, and checkin the connection when finished.
# File activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb, line 403 def with_connection unless conn = @thread_cached_conns[connection_cache_key(Thread.current)] conn = connection fresh_connection = true end yield conn ensure release_connection if fresh_connection end