Skip to content

feat: add custom transport support for Postgres and MySQL - #4281

Open
Benjscho wants to merge 2 commits into
transact-rs:mainfrom
Benjscho:feat/custom-socket-async-rw
Open

Benjscho wants to merge 2 commits into
transact-rs:mainfrom
Benjscho:feat/custom-socket-async-rw

Conversation

@Benjscho

Copy link
Copy Markdown

Add methods to PgConnection and MySqlConnection that accept pre-connected sockets implementing AsyncRead + AsyncWrite, enabling custom transport layers (vsock, QUIC, turmoil, SSH tunnels, etc.) without forking sqlx.

Per feedback on #4187, this uses AsyncRead + AsyncWrite traits instead of exposing the internal Socket trait. Two separate methods are provided for each runtime's trait set:

  • connect_with_custom_tokio(): accepts tokio::io::{AsyncRead, AsyncWrite}
  • connect_with_custom_futures(): accepts futures_io::{AsyncRead, AsyncWrite}

Also adds PoolOptions::connector() so pools can use custom transports:

PgPoolOptions::new()
    .connector(|options| async move {
        let socket = VsockStream::connect(addr).await?;
        PgConnection::connect_with_custom_tokio(socket, &options).await
    })
    .connect_with(options)
    .await?

TLS upgrade is negotiated automatically based on the connection options. No new public trait exposure. No behavioral changes to existing code.

Does your PR solve an issue?

No

Is this a breaking change?

No, it is additive with no behavior changes.

@Benjscho
Benjscho force-pushed the feat/custom-socket-async-rw branch 6 times, most recently from 073247c to 0704cd9 Compare May 25, 2026 21:44
@Benjscho
Benjscho marked this pull request as ready for review May 25, 2026 23:27
@Benjscho

Copy link
Copy Markdown
Author

@abonander would you be able to take a look at this when you have time? This was me taking a swing at addressing feedback on #4187.

#[cfg(feature = "_rt-tokio")]
pub struct TokioStream<S> {
inner: S,
read_buf: Vec<u8>,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't make sense to add yet another layer of buffering here. I know we've historically not focused on performance, but this would be an additional pessimization we don't really need.

I'm not against changing the Socket API to make this work (it could just be a combination of AsyncRead and AsyncWrite), however the problem is native_tls.

We have to wrap the stream in a type that implements the std::io::{Read, Write} traits, but we have to get the &mut Context inside for the .poll_*() calls. tokio-native-tls gets around this with unsafe code: https://docs.rs/tokio-native-tls/latest/src/tokio_native_tls/lib.rs.html#123-129

We could maybe use async-native-tls instead. It's essentially a copy-paste of tokio-native-tls (including the unsafe workaround) but the runtime-smol feature actually works for anything based on futures_io and doesn't directly depend on smol.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense! I'll take a look at getting that to work


/// Adapter that wraps a tokio [`AsyncRead`][tokio::io::AsyncRead] +
/// [`AsyncWrite`][tokio::io::AsyncWrite] into a [`Socket`] implementation.
#[cfg(feature = "_rt-tokio")]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's already rt-tokio and rt-async-io modules, I don't know why you chose to put this here with the extra #[cfg]s.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll clean this up

/// # Ok(())
/// # }
/// ```
pub fn connector<F, Fut>(mut self, connector: F) -> Self

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I already had a connector API I was working on in https://github.com/transact-rs/sqlx/blob/ab/pool-changes/sqlx-core/src/pool/connect.rs but I got bogged down in major changes to the pool internals. I could probably lift that part out into its own branch though.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That looks interesting, I could try to pull some of that in here, or if you want to pull it out into a branch that works too!

Comment on lines +60 to +63
pub async fn connect_raw_tokio<S>(
stream: S,
options: &MySqlConnectOptions,
) -> Result<Self, Error>

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should be inherent methods on MySqlConnectOptions and PgConnectOptions respectively, where all the other connect() methods already exist.

I'm not particularly enamored with the naming but I need to think about what I'd go with instead.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha, I'll move those

Comment thread sqlx-mysql/Cargo.toml
serde = { version = "1.0.219", optional = true }

tokio = { workspace = true, optional = true }
futures-io = { version = "0.3.32", optional = true }

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

futures-io should be promoted to a workspace dependency instead of repeating the version number everywhere.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll fix that

)
))]
#[sqlx_macros::test]
async fn it_connects_raw_tokio_with_tls() -> anyhow::Result<()> {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test shouldn't be necessary. If DATABASE_URL contains SSL options it should use TLS automatically.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I'll clean that up

Add methods to PgConnection and MySqlConnection that accept pre-connected
sockets implementing AsyncRead + AsyncWrite, enabling custom transport
layers (vsock, QUIC, turmoil, SSH tunnels, etc.) without forking sqlx.

Per maintainer feedback on transact-rs#4187, this uses AsyncRead + AsyncWrite traits
instead of exposing the internal Socket trait. Two separate methods are
provided for each runtime's trait set:

- connect_with_custom_tokio(): accepts tokio::io::{AsyncRead, AsyncWrite}
- connect_with_custom_futures(): accepts futures_io::{AsyncRead, AsyncWrite}

Also adds PoolOptions::connector() so pools can use custom transports:

    PgPoolOptions::new()
        .connector(|options| async move {
            let socket = VsockStream::connect(addr).await?;
            PgConnection::connect_with_custom_tokio(socket, &options).await
        })
        .connect_with(options)
        .await?

TLS upgrade is negotiated automatically based on the connection options.
No new public trait exposure. No behavioral changes to existing code.
@Benjscho
Benjscho force-pushed the feat/custom-socket-async-rw branch from 8a080b5 to c336a59 Compare November 19, 2026 17:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants