Writing large amounts of data to disk, only for databricks-sql-connector to then read it back in from disk, is incredibly inefficient.

It would be much more efficient to be able to provide a file-like object to use instead of a filepath. In that way a user could write the data to an in-memory io.BytesIO object instead of writing the data to disk.