mattn/go-sqlite3

go-sqlite3

Sponsors

This project is proudly sponsored by:

Latest stable version is v1.14 or later, not v2.

Description

A sqlite3 driver that conforms to the built-in database/sql interface.

Supported Golang version: See .github/workflows/go.yaml.

This package follows the official Golang Release Policy.

Overview

• go-sqlite3
• Description
  • Overview
• Installation
• API Reference
• Connection String
  • DSN Examples
• Features
  • Usage
  • Feature / Extension List
• Compilation
  • Android
• ARM
• Cross Compile
• Compiling
  • Linux
    • Alpine
    • Fedora
    • Ubuntu
  • macOS
  • Windows
  • Errors
• User Authentication
  • Compile
  • Usage
    • Create protected database
    • Password Encoding
      • Available Encoders
    • Restrictions
    • Support
    • User Management
      • SQL
        • Examples
      • *SQLiteConn
    • Attached database
• Extensions
  • Spatialite
• FAQ
• License
• Author

Installation

This package can be installed with the go get command:

go get github.com/mattn/go-sqlite3

go-sqlite3 is cgo package. If you want to build your app using go-sqlite3, you need gcc.

Important: because this is a CGO enabled package, you are required to set the environment variable CGO_ENABLED=1 and have a gcc compiler present within your path.

API Reference

API documentation can be found here.

Examples can be found under the examples directory.

Connection String

When creating a new SQLite database or connection to an existing one, with the file name additional options can be given. This is also known as a DSN (Data Source Name) string.

Options are append after the filename of the SQLite database. The database filename and options are separated by an ? (Question Mark). Options should be URL-encoded (see url.QueryEscape).

This also applies when using an in-memory database instead of a file.

Options can be given using the following format: KEYWORD=VALUE and multiple options can be combined with the & ampersand.

This library supports DSN options of SQLite itself and provides additional options.

Boolean values can be one of:

• 0 no false off
• 1 yes true on

DSN Examples

file:test.db?cache=shared&mode=memory

Features

This package allows additional configuration of features available within SQLite3 to be enabled or disabled by golang build constraints also known as build tags.

Click here for more information about build tags / constraints.

Usage

If you wish to build this library with additional extensions / features, use the following command:

go build -tags "<FEATURE>"

For available features, see the extension list. When using multiple build tags, all the different tags should be space delimited.

Example:

go build -tags "icu json1 fts5 secure_delete"

Feature / Extension List

Compilation

This package requires the CGO_ENABLED=1 environment variable if not set by default, and the presence of the gcc compiler.

If you need to add additional CFLAGS or LDFLAGS to the build command, and do not want to modify this package, then this can be achieved by using the CGO_CFLAGS and CGO_LDFLAGS environment variables.

Android

This package can be compiled for android. Compile with:

go build -tags "android"

For more information see #201

ARM

To compile for ARM use the following environment:

env CC=arm-linux-gnueabihf-gcc CXX=arm-linux-gnueabihf-g++ \
    CGO_ENABLED=1 GOOS=linux GOARCH=arm GOARM=7 \
    go build -v 

Additional information:

• #242
• #504

Cross Compile

This library can be cross-compiled.

In some cases you are required to the CC environment variable with the cross compiler.

Cross Compiling from macOS

The simplest way to cross compile from macOS is to use xgo.

Steps:

• Install musl-cross (brew install FiloSottile/musl-cross/musl-cross).
• Run CC=x86_64-linux-musl-gcc CXX=x86_64-linux-musl-g++ GOARCH=amd64 GOOS=linux CGO_ENABLED=1 go build -ldflags "-linkmode external -extldflags -static".

Please refer to the project's README for further information.

Compiling

Linux

To compile this package on Linux, you must install the development tools for your linux distribution.

To compile under linux use the build tag linux.

go build -tags "linux"

If you wish to link directly to libsqlite3 then you can use the libsqlite3 build tag.

go build -tags "libsqlite3 linux"

Alpine

When building in an alpine container run the following command before building:

apk add --update gcc musl-dev

Fedora

sudo yum groupinstall "Development Tools" "Development Libraries"

Ubuntu

sudo apt-get install build-essential

macOS

macOS should have all the tools present to compile this package. If not, install XCode to add all the developers tools.

Required dependency:

brew install sqlite3

For macOS, there is an additional package to install which is required if you wish to build the icu extension.

This additional package can be installed with homebrew:

brew upgrade icu4c

To compile for macOS on x86:

go build -tags "darwin amd64"

To compile for macOS on ARM chips:

go build -tags "darwin arm64"

If you wish to link directly to libsqlite3, use the libsqlite3 build tag:

# x86 
go build -tags "libsqlite3 darwin amd64"
# ARM
go build -tags "libsqlite3 darwin arm64"

Additional information:

• #206
• #404

Windows

To compile this package on Windows, you must have the gcc compiler installed.

1. Install a Windows gcc toolchain.
2. Add the bin folder to the Windows path, if the installer did not do this by default.
3. Open a terminal for the TDM-GCC toolchain, which can be found in the Windows Start menu.
4. Navigate to your project folder and run the go build ... command for this package.

For example the TDM-GCC Toolchain can be found here.

Errors

• Compile error: can not be used when making a shared object; recompile with -fPIC

  When receiving a compile time error referencing recompile with -FPIC then you are probably using a hardend system.

  You can compile the library on a hardend system with the following command.

  go build -ldflags '-extldflags=-fno-PIC'
  

  More details see #120

• Can't build go-sqlite3 on windows 64bit.

  Probably, you are using go 1.0, go1.0 has a problem when it comes to compiling/linking on windows 64bit. See: #27

• go get github.com/mattn/go-sqlite3 throws compilation error.

  gcc throws: internal compiler error

  Remove the download repository from your disk and try re-install with:

  go install github.com/mattn/go-sqlite3
  

User Authentication

This is deprecated

This package supports the SQLite User Authentication module.

Compile

To use the User authentication module, the package has to be compiled with the tag sqlite_userauth. See Features.

Usage

Create protected database

To create a database protected by user authentication, provide the following argument to the connection string _auth. This will enable user authentication within the database. This option however requires two additional arguments:

• _auth_user
• _auth_pass

When _auth is present in the connection string user authentication will be enabled and the provided user will be created as an admin user. After initial creation, the parameter _auth has no effect anymore and can be omitted from the connection string.

Example connection strings:

Create an user authentication database with user admin and password admin:

file:test.s3db?_auth&_auth_user=admin&_auth_pass=admin

Create an user authentication database with user admin and password admin and use SHA1 for the password encoding:

file:test.s3db?_auth&_auth_user=admin&_auth_pass=admin&_auth_crypt=sha1

Password Encoding

The passwords within the user authentication module of SQLite are encoded with the SQLite function sqlite_cryp. This function uses a ceasar-cypher which is quite insecure. This library provides several additional password encoders which can be configured through the connection string.

The password cypher can be configured with the key _auth_crypt. And if the configured password encoder also requires an salt this can be configured with _auth_salt.

Available Encoders

• SHA1
• SSHA1 (Salted SHA1)
• SHA256
• SSHA256 (salted SHA256)
• SHA384
• SSHA384 (salted SHA384)
• SHA512
• SSHA512 (salted SHA512)

Restrictions

Operations on the database regarding user management can only be preformed by an administrator user.

Support

The user authentication supports two kinds of users:

• administrators
• regular users

User Management

User management can be done by directly using the *SQLiteConn or by SQL.

SQL

The following sql functions are available for user management:

These functions will return an integer:

• 0 (SQLITE_OK)
• 23 (SQLITE_AUTH) Failed to perform due to authentication or insufficient privileges

Examples

// Autheticate user
// Create Admin User
SELECT auth_user_add('admin2', 'admin2', 1);

// Change password for user
SELECT auth_user_change('user', 'userpassword', 0);

// Delete user
SELECT user_delete('user');

*SQLiteConn

The following functions are available for User authentication from the *SQLiteConn:

Attached database

When using attached databases, SQLite will use the authentication from the main database for the attached database(s).

Extensions

If you want your own extension to be listed here, or you want to add a reference to an extension; please submit an Issue for this.

Spatialite

Spatialite is available as an extension to SQLite, and can be used in combination with this repository. For an example, see shaxbee/go-spatialite.

extension-functions.c from SQLite3 Contrib

extension-functions.c is available as an extension to SQLite, and provides the following functions:

• Math: acos, asin, atan, atn2, atan2, acosh, asinh, atanh, difference, degrees, radians, cos, sin, tan, cot, cosh, sinh, tanh, coth, exp, log, log10, power, sign, sqrt, square, ceil, floor, pi.
• String: replicate, charindex, leftstr, rightstr, ltrim, rtrim, trim, replace, reverse, proper, padl, padr, padc, strfilter.
• Aggregate: stdev, variance, mode, median, lower_quartile, upper_quartile

For an example, see dinedal/go-sqlite3-extension-functions.

FAQ

• Getting insert error while query is opened.

  You can pass some arguments into the connection string, for example, a URI. See: #39

• Do you want to cross compile? mingw on Linux or Mac?

  See: #106 See also: http://www.limitlessfx.com/cross-compile-golang-app-for-windows-from-linux.html

• Want to get time.Time with current locale

  Use _loc=auto in SQLite3 filename schema like file:foo.db?_loc=auto.

• Can I use this in multiple routines concurrently?

  Yes for readonly. But not for writable. See #50, #51, #209, #274.

• Why I'm getting no such table error?

  Why is it racy if I use a sql.Open("sqlite3", ":memory:") database?

  Each connection to ":memory:" opens a brand new in-memory sql database, so if the stdlib's sql engine happens to open another connection and you've only specified ":memory:", that connection will see a brand new database. A workaround is to use "file::memory:?cache=shared" (or "file:foobar?mode=memory&cache=shared"). Every connection to this string will point to the same in-memory database.

  Note that if the last database connection in the pool closes, the in-memory database is deleted. Make sure the max idle connection limit is > 0, and the connection lifetime is infinite.

  For more information see:

  • #204
  • #511
  • https://www.sqlite.org/sharedcache.html#shared_cache_and_in_memory_databases
  • https://www.sqlite.org/inmemorydb.html#sharedmemdb

• Reading from database with large amount of goroutines fails on OSX.

  OS X limits OS-wide to not have more than 1000 files open simultaneously by default.

  For more information, see #289

• Trying to execute a . (dot) command throws an error.

  Error: Error: near ".": syntax error Dot command are part of SQLite3 CLI, not of this library.

  You need to implement the feature or call the sqlite3 cli.

  More information see #305.

• Error: database is locked

  When you get a database is locked, please use the following options.

  Add to DSN: cache=shared

  Example:

  db, err := sql.Open("sqlite3", "file:locked.sqlite?cache=shared")
  

  Next, please set the database connections of the SQL package to 1:

  db.SetMaxOpenConns(1)
  

  For more information, see #209.

Contributors

Code Contributors

This project exists thanks to all the people who [contribute].

Financial Contributors

Become a financial contributor and help us sustain our community. [Contribute here].

Individuals

Organizations

Support this project with your organization. Your logo will show up here with a link to your website. [Contribute]

License

MIT: http://mattn.mit-license.org/2018

sqlite3-binding.c, sqlite3-binding.h, sqlite3ext.h

The -binding suffix was added to avoid build failures under gccgo.

In this repository, those files are an amalgamation of code that was copied from SQLite3. The license of that code is the same as the license of SQLite3.

Author

Yasuhiro Matsumoto (a.k.a mattn)

G.J.R. Timmer