v0.1.662026-11-19
pgdog v0.1.66 更新:性能与跨分片查询能力双提升
本次更新在性能优化和跨分片查询支持上都有明显进展。
性能方面,跨分片 CTE 与子查询改用 unnest 传递数据,并引入二进制编码,跨分片查询的开销进一步降低。CommandComplete 消息解析、会话参数检查等高频路径也做了加速,同时清理了未使用的构建依赖。
跨分片查询支持大幅增强:新增 booland、boolor、every、bitand、bitor 聚合函数;子查询现在支持全部 Postgres 内置类型和 ARRAY 类型;LIMIT/OFFSET 支持整数运算;SELECT 配合聚合与 HAVING 时的列别名也能正确处理。
Bug 修复覆盖多个场景:WHERE 子句中多个跨分片子查询结果混淆、CTE 别名列被误用于分片路由、仅返回常量的子查询处理不当、仅含单个 schema 的 search_path 解析错误、直通认证在 pooler 启动时的竞态条件、跨分片键更新失败时多发 ReadyForQuery、以及轮询模式下主机被封禁时负载分配不均等问题均已修复。此外,事务模式下 SET ROLE 和 SET SESSION AUTHORIZATION 现在能正确处理,schema 同步也改用 CREATE IF NOT EXISTS,对象创建更加幂等。
v0.1.652026-11-12
pgdog v0.1.65 更新:性能与跨分片查询能力双提升
性能方面,查询热路径进一步优化,减少了内存分配、CPU 指令和互斥锁争用;跨分片子查询改用 unnest() 传参,性能大幅改善。
功能上,跨分片 GROUP BY 现支持聚合 SELECT 之外的列、HAVING 子句,且 LIMIT/OFFSET 会在聚合后正确应用;新增跨分片 CTE、子查询派生表、ORDER BY 的 NULLS FIRST/NULLS LAST 支持;逻辑复制现可单独迁移表,不再局限于重新分片场景。
修复了多个问题:无聚合函数的跨分片 GROUP BY 返回错误结果、透传认证令牌缓存过期条目阻塞复用、prefer_primary 在只读集群误路由到缺失主库、INSERT [...] SELECT 行路由错误、聚合函数溢出导致 panic、查询取消偶发失败等。
v0.1.642026-11-05
pgdog v0.1.64 更新:性能与稳定性双提升
这版在性能和稳定性上都有实打实的改进。性能方面,release 构建启用了 PGO(Profile-Guided Optimization),x86 平台提速约 5%,ARM64 更是提升 30%。
修复的问题不少,几个比较关键:自动模式下的两阶段提交遇到多行 INSERT 时无法正确提交;Rails 健康检查用的空查询(;)会在失败事务中被卡住;查询取消请求若晚于查询完成到达 Postgres,会在连接间泄漏;只读用户无法发现主键,导致分片部署中自动注入主键失效;UNLISTEN 此前完全无效,现在能正确退订所有频道;连续快速 RELOAD 可能把"pool is offline"错误传给已连接的客户端;副本延迟公式在用户修改 timezone 后会出错,现在延迟查询中的时间函数统一使用 AS TIME ZONE 'UTC';readwritesplit = "excludeprimary" 在没有副本时仍会读主库;跨分片查询的扩展协议中注入 Describe,用返回的 RowDescription 解码行,避免 resharding 时 integer 升级为 bigint 造成的数据类型不匹配。
功能上,跨分片 SELECT 正确处理 count()::numeric 类型转换;可通过查询注释(如 / pgdoglsn: 0/0 / SELECT [...])按 LSN 过滤负载均衡中的副本;users.toml 新增 readwritesplit 设置;聚合部分支持 HAVING 子句;支持对 inet、money、cidr 类型排序和聚合。
安全方面,Docker 镜像重新加入了自签名证书,仅供测试用,生产环境请勿使用,安全扫描器报错可忽略。
v0.1.632026-11-02
Features
- Add support for recursive cross-shard subqueries (i.e., subquery inside another subquery) @jkaczman
- Add support for INSERT INTO [...] SELECT [...] FROM subqueries @jkaczman
- Add support for handling subqueries in right side of boolean expression in a WHERE clause @jkaczman
- Breaking preparedstatementslimit is now set to 5,000 by default. This is to prevent statement leaks when that setting is not set and the pg client driver does not use prepared statements optimally (e.g., tokio-postgres). @levkk
- Add a bunch of CLI tools to the production Docker image to help with debugging (e.g., netstat, dig, etc.)
- Add minconnections OTEL/OpenMetrics gauge which tracks minpool_size for each connection pool @meskill
- Make SIGTERM behave like SIGINT, preventing common misuse in cloud-native environments @meskill
- Cross-shard ORDER BY now supports CASE expressions @jkaczman
- Partial support for HAVING in cross-shard aggregates @murex971
- Support cumulative temporailty for OTEL metrics, allowing to export those metrics to servers that don't support incremental measurements, e.g., GCP @jkaczman
- Parse URI on TLS certificates along with dnsName to be used as identity for client-to-PgDog mTLS @eidam @levkk
Performance
- Query router performance improvements (5-12% improvement in microbenchmarks). @jkaczman
- Replace query cache mutex with a rwlock, decreasing contention on the AST cache. The cache now uses CLOCK (second chance) eviction to enforce the LRU policy. @jkaczman
- Improve cross-shard INSERT handling by grouping tuples that belong to the same shard into a single statement. @jkaczman
- Skip query routing logic in non-sharded single primary/replicas-only clusters. CPU utilization reduced by 1.3%. @jkaczman
- Make PgDog production Docker images 35% smaller @meskill
- Remove redundant clock reads in the hot path @jkaczman
Bug fixes
- Make SET more Postgres-compliant: return errors on incorrect parameter values, return correct ParameterStatus messages, handle RESET correctly for DateStyle and other parameters that have connection defaults, correctly return multiple CommandComplete messages for multi-query SET statements.
- minpoolsize was incorrectly rejected on [[users]] entries when passthrough_auth was enabled @meskill
- Logical replication couldn't handle tables with primary keys that cover all table columns @levkk
- Extended protocol error recovery had an edge case that could cause PgDog to stall @levkk @frangz
- Sharding key updates were incorrectly blocked to omnisharded tables wherie they were safe @levkk
- [Enterprise] Cancelled replication tasks were incorrectly tracked as completed @meskill
- Some resharding tasks could hang on cancellation @meskill
- Some extended protocol messages (e.g., Describe) were incorrectly routed in cross-shard transactions @meskill
- PgDog would crash if it would run out of available open file descriptions. Correct behavior is to return an error and continue operations. @meskill
- Reject unsupported cross-shard aggregates with HAVING clause instead of returning wrong results @murex971
- Correctly parse -c startup parameters in Startup message @meskill
- Cross-shard INSERT transaction using twophasecommit_auto would not automatically commit @meskill
- Don't disconnect clients that attempt to use LISTEN/NOTIFY when pubsubchannel_size is 0 @murex971
- Correctly stick pg catalog omnisharded queries to one shard, allowing tools like pg_dump to work correctly with sharded databases @meskill
- GROUP BY 0 and other invalid values would crash PgDog @murex971
Security
- Prevent clients directly executing PgDog-prepared statements by internal name. This only affected users that were able to authenticate to PgDog, so multi-tenant publicly accessible deployments should consider upgrading. @levkk @ewhauser
Code quality
- Subquery test suite runs in parallel for all DML queries @jkaczman
- Cleanup transaction handling to re-use the same simulated responses code @levkk
- Use pgstatmonitor extension in tests to validate correct statement routing @jkaczman
- Move to self-hosted metal instances for performance regression testing @jkaczman
- Updated to latest Tokio @meskill
- Use instruction count instead of latency for microbenchmarks @jkaczman
Full Changelog: https://github.com/pgdogdev/pgdog/compare/v0.1.62...v0.1.63
v0.1.622026-10-16
Features
- Basic support for cross-shard subqueries @jkaczman
- Add auth_type config to user entries in users.toml @levkk @BABTUNA
- Use replication origin to track replication progress during resharding. This makes it durable and allows us to recover progress after a crash (still a TODO). @meskill
- Add row-level metrics for DML/DDL queries @murex971
- Ensure non-deterministic functions like now() return consistent results in cross-shard DELETEs @jkaczman
- Support parsing custom logical messages to handle DDL replication @jkaczman
Bug fixes
- pgtryadvisory_lock handling was optimistic / did not account for lock not being actually available. PgDog will only pin the session now if the lock was actually acquired. @levkk
- DISCARD ALL returned wrong command tag, breaking JDBC driver prepared statements handling @levkk
- SET now correctly overrides SET LOCAL inside a transaction @levkk
- RESET now correctly restores startup parameters @levkk
- ROLLBACK no longer undo's a RESET from previous transaction @levkk
- Invalid parameter settings would leave server connection in a partial/dirty state @levkk
- Parameter names were incorrectly escaped @levkk
- SET LOCAL [...] TO DEFAULT would be incorrectly treated as SET @anna-poutanen
- PgDog now compiles under FreeBSD @mahlonsmith
- COPY identifiers (table name and column names) are now correctly escaped during resharding @BABTUNA
- Correctly handle ABORT and END when query parser is off. This was regression introduced in v0.1.60. @levkk @valanor-vaelmir
- Correctly detect sharding key in join clause @levkk @PeterZaitsev
- Correctly check that each shard committed the transaction before returning success during resharding @jkaczman
- Unsupported agggregate functions now return an ERROR instead of FATAL @levkk @PeterZaitsev
- Fix avg() scale for cross-shard numerics @murex971 @PeterZaitsev
Performance
- Reduce memory usage of prepared statements cache after bursts of new statements are added and removed. This affects drivers like tokio-postgres, go/pq, Postgrex and others that prepare one-off prepared statements to execute queries
Code quality
- Improve replication benchmark @meskill
- Microbenchmarks for our query engine @jkaczman
Full Changelog: https://github.com/pgdogdev/pgdog/compare/v0.1.61...v0.1.62
v0.1.612026-10-09
Performance
- Improve resharding COPY read performance by over 10x on network-bound storage databases (e.g., RDS) with large toasted tables @jkaczman
- Cache query ASTs regardless of comments, improving query parse time for queries that use comments for routing @sgrif
- Use jemalloc in our query parser, decreasing memory usage for deployments using long queries @levkk
- Improve schema loading times at PgDog startup by a factor of 10 @levkk @ariel-lahiany
- Improve resharding replication performance by 2x by switching to async shard writes @jkaczman
Bug fixes
- LISTEN and NOTIFY didn't escape identifiers, breaking pub/sub for channel names and payloads containing non-escaped data @mhenrixon
- Passthrough auth would block pool until configuration was reloaded if user connected with a bad password. We now check the password before adding the user into the in-memory config. @levkk
- Backend connection recovery would commit implicit transactions for pipelined requests @murex971
- Cross-shard ORDER BY would use wrong column if it was aliased in the target list @jkaczman
- Account for WAL timeline to detect new primary after failover; tools like Patroni don't terminate failed primaries, so PgDog was confused when it was able to connect to one @levkk @roshangara
- LISTEN/NOTIFY broker connection would not restart if terminated by Postgres @levkk @alphec
- Fix crash if multi-statement query contained RESET ALL @levkk @roshangara
- Regression in LIMIT not working without ORDER BY in cross-shard queries @levkk @nsavvide
- Correctly handle SET LOCAL outside of transactions @murex971
- CREATE TEMP SEQUENCE now correctly pins connections @murex971
- Fix off-by-one error in sharding replication table LSNs @jkaczman
- Don't rewrite client requests before mirroring, allowing mirrors to use a different sharding configuration @sgrif
- Fix configuration reloads blocking traffic to existing databases temporarily while they run a health check if requirehealthcheckon_discovery was enabled @frodsan
- Generate direct-to-shard and cross-shard variants for queries containing aggregates/ordering/limit/offset/etc. to make sure we don't rewrite the query if it uses direct-to-shard routing. @murex971
- pgadvisoryunlock(NULL) incorrectly released all advisory locks @dipeshbabu
- Correctly handle parameter casting for UUID parameters in query routing for sharded deployments @sgrif
- Correctly handle RESET ALL @dipeshbabu
Features
- Allow to configure auth_type on users in users.toml
- [Breaking] Plugins now have to process one statement at a time @sgrif
- [Experimental] Support for RDS IAM passthrough authentication. PgDog attempts to connect to RDS with the token provided by the user, and if successful, caches that token for 15 minutes. This works reasonably well if clients use IAM tokens correctly (i.e., they cache them for 15 minutes, too) @levkk
- [Breaking] For globally distributed sequences (EE-only), include the schema name into the sequence @murex971
- Validate foreign key constraints after resharding is complete @meskill
- Transactions now lazily connect to shards as needed. This allows direct-to-shard explicit transactions not to connect to all shards at all times, improving connection pool efficiency. @levkk
- Cross-shard UPDATE and SELECT queries that use non-deterministic functions (e.g., now(), current_timestamp()) now read/write consistent values across all shards (generated by PgDog instead of the database) @jkaczman
- Show accurate memory usage for AST cache using Postgres arena allocator statistic @sgrif
- Wait for resharding tasks to shutdown gracefully on PgDog shutdown @meskill
- Send replication state to control plane (EE) @meskill
Code quality
- Sharded our CI, which is now 5 times faster @jkaczman
- Benchmark each commit using Bencher to prevent performance regressions @jkaczman
Note for contributors
We stopped accepting pull requests on the repository due to large influx of AI-generated submissions. PgDog still accepts bug reports, feature requests, and fixes, as long as those are submitted via GitHub issues instead. If you want to provide code, please send us a branch from your own fork. We will probably rewrite it to match our code quality standards, but we'll credit you for the contribution.
🙏 ❤️
Full Changelog: https://github.com/pgdogdev/pgdog/compare/v0.1.60...v0.1.61
v0.1.602026-09-24
Performance
* 40% faster handling of SELECT in sharded queries by removing redundant AST walks @jkaczman
Bugs
* 0.0 and -0.0 hashed to the same value, which broke DISTINCT, count() and other cross-shard functions @MaxFreedomPollard
sessionreplication_role should be set via SET command @levkk
* interval type wasn't correctly parsed for some edge cases in cross-shard aggregates @dipeshbabu
* stddev / variance failed on some NULL values in cross-shard aggregates @dipeshbabu
* count() wouldn't preserve original input type / would cast everything to bigint @dipeshbabu
* NULL was incorrectly handled in array elements in cross-shard queries @dipeshbabu
* timestamp(tz) infinite values were not correctly parsed in cross-shard queries @dipeshbabu
* timestamp(tz) fractional time wasn't correctly preserved in cross-shard queries @dipeshbabu
* jemalloc would crash on ARM targets with page size >= 16k @dipeshbabu
* client/server TLS wouldn't load the whole chain, so client keys signed by intermediates were rejected @dipeshbabu
* don't run 2pc on direct-to-shard write transactions @rlittlefield
* connection pool statistics are preserved on RELOAD @murex971
* handle numeric infinite values correctly in cross-shard queries @dipeshbabu
* fully-qualified columns were skipped in DISTINCT ON @jkaczman
* sequence names are quoted to make sure they work with all table names @dipeshbabu
don't drop SET checkfunction_bodies in schema sync @meskill
* pg deparser wasn't handling timezone operations correctly @meskill
* correctly route advisory lock calls in sharded deployments & handle the two parameter variant @jkaczman
correctly track pgcatalog.pgadvisory fully qualified advisory lock calls @dipeshbabu
Code quality
* migrate replication/resharding to new tasks API @meskill
uniqueid and insert_splits cleanup @sgrif
* auto retries to some flaky spots in CI @jkaczman
New Contributors
* @MaxFreedomPollard made their first contribution in https://github.com/pgdogdev/pgdog/pull/1507
Full Changelog: https://github.com/pgdogdev/pgdog/compare/v0.1.59...v0.1.60
v0.1.592026-09-17
Bug fixes
- [Breaking] Manual bans are now carried over to new connection pools after RELOAD. They were previously discarded, allowing traffic to flow through prematurely after configuration reload. Breaking because this changes existing behavior and may require operators to adjust migration scripts to issue a separate UNBAN command after RELOAD. @levkk
- Schema reload was not triggered on certain DDL statements, e.g., CREATE TYPE, which caused incorrect RowDescription messages sent to clients for direct-to-shard and cross-shard queries which touched those new types. @levkk
- 2pc WAL checkpointer could cause WAL corruption by removing segments necessary for recovery. Fixed by storing relationships between segments containing data about the same transaction, preventing dependent segments from being removed prematurely. @levkk
- SCRAM authentication could cause DoS because it was moved to background Tokio threads, which were unbounded. Added background_workers setting (default 0, disabling this whole feature) to limit that concurrency. Moved SCRAM auth back to the Tokio async worker pool. @levkk
- Fix segfault in query normalization (only affected the Enterprise edition) caused by a bug in our parser. Added pgrawparse::normalize to our regression test suite to prevent recurrences. @levkk
- Setting connection_recovery to drop would cause connection churn for connection drivers using prepared statements. Fixed by ensuring we don't drop connections in this scenario. @levkk
- Disable triggers on destination shards during resharding (by setting replicasessionrole to replica). This prevents triggers from running twice and also ensures we can copy tables that use foreign keys and which delete / update relationships during resharding. @meskill
- Prevent deadlock in extended protocol pipelines when client sends a Describe out of specific order @murex971
- Correctly handle omnisharded to sharded joins where the sharding key is present in omnisharded tables @ygxio @levkk
- Connections using LISTEN would incorrectly handle that command sent via extended protocol (returning ReadyForQuery prematurely) @murex971
- When using passthrough auth, reloading the config would not carry over any user-specific settings configured in users.toml @mehcode
- Role detection (role = "auto") could cause deadlock in pool for read-only clusters that served write transactions (e.g., BEGIN, not BEGIN READ ONLY) @levkk
- Query router wouldn't handle nested typecasts correctly for shard key detection @sgrif
Performance
- Improve performance of resharding copy data phase when using multiple replicas by more fairly distributing workloads between source databases @meskill
Features
- Support LIMIT and OFFSET in cross-shard statements sent via PREPARE and EXECUTE @jkaczman
- Support cross-account RDS IAM authentication by using STS assume-role @lyupan-cb
- Add totalwaittime and avgwaittime to OpenMetrics (and OTEL) @rimrakhimov
- Support Postgres foreign keys during resharding by disabling Postgres triggers during replication. This also ensures that triggers on the destination shards don't run, potentially causing double data insertion. @meskill
- Inject correct time (e.g. timestamp, timestamptz, etc.) and UUID values in cross-shard and omnisharded INSERT queries, to make sure the values are identical on all shards. This replaces database-generated defaults without changing the schema. Enable by turning on rewrite.nondeterministicfunctions = "rewrite". @jkaczman
- Add checkout_timeout metric to OpenMetrics (and OTEL) and admin database (SHOW STATS) which shows how many times clients hit the error for a particular pool @levkk
Code quality
- Fix typo in config example @joshuaharry
- Refactor resharding copy data code to use the tasks API @meskill
- Cleanup of omnisharded table handling during copy data phase @meskill
- Remove redundant check in shard key update flow @sgrif
- CI now uses Postgres 18 (used to be 16) @jkaczman
- Make some tests more deterministic instead of relying on timing @dipeshbabu @ziomarco
New Contributors
* @joshuaharry made their first contribution in https://github.com/pgdogdev/pgdog/pull/1532
* @lyupan-cb made their first contribution in https://github.com/pgdogdev/pgdog/pull/1525
* @rimrakhimov made their first contribution in https://github.com/pgdogdev/pgdog/pull/1528
* @mehcode made their first contribution in https://github.com/pgdogdev/pgdog/pull/1555
* @dipeshbabu made their first contribution in https://github.com/pgdogdev/pgdog/pull/1558
* @ziomarco made their first contribution in https://github.com/pgdogdev/pgdog/pull/1570
Full Changelog: https://github.com/pgdogdev/pgdog/compare/v0.1.58...v0.1.59
v0.1.582026-09-10
Features
- Database-specific TLS configuration (incl. mTLS), allowing different TLS certificates per host in pgdog.toml @rlittlefield
- Correct handling of DISCARD, incl. DISCARD TEMP and DISCARD ALL @murex971
- Add FORCE_RELOAD admin command which guarantees all subsequent queries are sent to new connection pools (vs. RELOAD which is async) @jkaczman
- Query engine hooks for pgdog.nextval function (only available in EE) to generate globally unique, monotonically increasing integers with no gaps (using Raft in the control plane) @levkk
Bugs
- Allow omnisharded reads with a shard comment routing inside cross-shard write transactions @rlittlefield
- Connections were unncessarily dropped and re-created when a replica was added (or removed) in pgdog.toml @jkaczman
- Correctly handle omnisharded writes inside CTEs @rlittlefield
Code quality
- Added CHANGELOG.md and CHANGELOG-ENTERPRISE.md to track changes to open source and enterprise edition @levkk
Full Changelog: https://github.com/pgdogdev/pgdog/compare/v0.1.57...v0.1.58
v0.1.572026-09-03
Performance
- SCRAM client authentication (which uses expensive crypto) is now pushed to background threads, which will avoid blocking other clients from running queries during connection storms
Features
- Add listenbacklog configuration setting to control how many new, concurrent_, TCP connections PgDog will allow until they are silently rejected @ryderhwang
- DISCARD now clears client's prepared statement cache, advisory locks & notification channels @murex971
- CREATE TEMP TABLE now pins the client connection until the table is dropped or the client disconnects @sgrif
- Count and show number of idle-in-transaction timeouts as a pool statistic @murex971
- Add support for SCRAM-SHA-256-PLUS client authentication @abnegate
- Add banned gauge to OpenMetrics and OTEL indicating which database is currently blocked from reads by the load balancer @ChrisJr404
Bugs
- Fix cross-shard variance() crash when empty/null results returned by shards @sgrif
- Prepared statements prepared via simple protocol now respect preparedstatementsttl @jkaczman
Code quality
- Refactor resharding task queue to use more types @meskill
- Clean up resharding logging @levkk
- Refactor the query engine code to support cross-shard subqueries and CTEs @jkaczman
New Contributors
* @ryderhwang made their first contribution in https://github.com/pgdogdev/pgdog/pull/1452
* @abnegate made their first contribution in https://github.com/pgdogdev/pgdog/pull/1457
* @mvidaurre made their first contribution in https://github.com/pgdogdev/pgdog/pull/1483
* @ChrisJr404 made their first contribution in https://github.com/pgdogdev/pgdog/pull/1443
Full Changelog: https://github.com/pgdogdev/pgdog/compare/v0.1.56...v0.1.57