testfixtures
Warning: this package will wipe the database data before loading the fixtures! It is supposed to be used on a test database. Please, double check if you are running it against the correct database.
TIP: There are options not described in this README page. It’s recommended that you also check the documentation.
Writing tests is hard, even more when you have to deal with an SQL database. This package aims to make writing functional tests for web apps written in Go easier.
Basically this package mimics the “Ruby on Rails’ way” of writing tests for database applications, where sample data is kept in fixtures files. Before the execution of every test, the test database is cleaned and the fixture data is loaded into the database.
The idea is running tests against a real database, instead of relying in mocks, which is boring to setup and may lead to production bugs not being caught in the tests.
comments.yml
id: 1 post_id: 1 content: A comment… author_name: John Doe author_email: john@doe.com created_at: 2020-12-31 23:59:59 updated_at: 2020-12-31 23:59:59
id: 2 post_id: 2 content: Another comment… author_name: John Doe author_email: john@doe.com created_at: 2020-12-31 23:59:59 updated_at: 2020-12-31 23:59:59
…
An YAML object or array will be converted to JSON. It will be stored on a native
JSON type like JSONB on PostgreSQL & CockroachDB or as a TEXT or VARCHAR column on other
databases.
```yml
- id: 1
post_attributes:
author: John Due
author_email: john@due.com
title: "..."
tags:
- programming
- go
- testing
post: "..."
Binary columns can be represented as hexadecimal strings (should start with 0x
):
- id: 1
binary_column: 0x1234567890abcdef
If you need to write raw SQL, probably to call a function, prefix the value
of the column with RAW=
:
- id: 1
uuid_column: RAW=uuid_generate_v4()
postgis_type_column: RAW=ST_GeomFromText('params...')
created_at: RAW=NOW()
updated_at: RAW=NOW()
Your tests would look like this:
package myapp
import (
"database/sql"
_ "github.com/lib/pq"
"github.com/go-testfixtures/testfixtures/v3"
)
var (
db *sql.DB
fixtures *testfixtures.Loader
)
func TestMain(m *testing.M) {
var err error
// Open connection to the test database.
// Do NOT import fixtures in a production database!
// Existing data would be deleted.
db, err = sql.Open("postgres", "dbname=myapp_test")
if err != nil {
...
}
fixtures, err = testfixtures.New(
testfixtures.Database(db), // You database connection
testfixtures.Dialect("postgres"), // Available: "postgresql", "timescaledb", "mysql", "mariadb", "sqlite" and "sqlserver"
testfixtures.Directory("testdata/fixtures"), // The directory containing the YAML files
)
if err != nil {
...
}
os.Exit(m.Run())
}
func prepareTestDatabase() {
if err := fixtures.Load(); err != nil {
...
}
}
func TestX(t *testing.T) {
prepareTestDatabase()
// Your test here ...
}
func TestY(t *testing.T) {
prepareTestDatabase()
// Your test here ...
}
func TestZ(t *testing.T) {
prepareTestDatabase()
// Your test here ...
}
Alternatively, you can use the Files
option, to specify which
files you want to load into the database:
fixtures, err := testfixtures.New(
testfixtures.Database(db),
testfixtures.Dialect("postgres"),
testfixtures.Files(
"fixtures/orders.yml",
"fixtures/customers.yml",
),
)
if err != nil {
...
}
With Paths
option, you can specify the paths that fixtures will load
from. Path can be directory or file. If directory, we will search YAML files
in it.
fixtures, err := testfixtures.New(
testfixtures.Database(db),
testfixtures.Dialect("postgres"),
testfixtures.Paths(
"fixtures/orders.yml",
"fixtures/customers.yml",
"common_fixtures/users"
),
)
if err != nil {
...
}
test_case1.yml
posts:
id: 1 post_id: 1 content: A comment… author_name: John Doe author_email: john@doe.com created_at: 2020-12-31 23:59:59 updated_at: 2020-12-31 23:59:59
id: 2 post_id: 2 content: Another comment… author_name: John Doe author_email: john@doe.com created_at: 2020-12-31 23:59:59 updated_at: 2020-12-31 23:59:59
comments:
id: 1 post_id: 1 content: Post 1 comment 1 author_name: John Doe author_email: john@doe.com created_at: 2016-01-01 12:30:12 updated_at: 2016-01-01 12:30:12
…
## Security check
In order to prevent you from accidentally wiping the wrong database, this
package will refuse to load fixtures if the database name (or database
filename for SQLite) doesn't contains "test". If you want to disable this
check, use:
```go
testfixtures.New(
...
testfixtures.DangerousSkipTestDatabaseCheck(),
)
Disable cleanup
If you want to disable cleanup, you can also do like below. This is usually not recommended, and should be used mostly for debugging.
testfixtures.New(
...
testfixtures.DangerousSkipCleanupFixtureTables(),
)
Sequences
For PostgreSQL and MySQL/MariaDB, this package also resets all sequences to a high number to prevent duplicated primary keys while running the tests. The default is 10000, but you can change that with:
testfixtures.New(
...
testfixtures.ResetSequencesTo(10000),
)
Or, if you want to skip the reset of sequences entirely:
testfixtures.New(
...
testfixtures.SkipResetSequences(),
)
Force DELETE FROM ...
on ClickHouse
By default, when using ClickHouse, this library will use TRUNCATE ...
to
clean the database. If you want to force the use of DELETE FROM ...
you can
do it by doing:
testfixtures.New(
...
testfixtures.ClickhouseUseDeleteFrom(),
)
Compatible databases
PostgreSQL / TimescaleDB / CockroachDB
This package has three approaches to disable foreign keys while importing fixtures for PostgreSQL databases:
With DISABLE TRIGGER
This is the default approach. For that use:
testfixtures.New(
...
testfixtures.Dialect("postgres"), // or "timescaledb"
)
With the above snippet this package will use DISABLE TRIGGER
to temporarily
disabling foreign key constraints while loading fixtures. This work with any
version of PostgreSQL, but it is required to be connected in the database
as a SUPERUSER. You can make a PostgreSQL user a SUPERUSER with:
ALTER USER your_user SUPERUSER;
With ALTER CONSTRAINT
This approach don’t require to be connected as a SUPERUSER, but only work with PostgreSQL versions >= 9.4. Try this if you are getting foreign key violation errors with the previous approach. It is as simple as using:
testfixtures.New(
...
testfixtures.Dialect("postgres"),
testfixtures.UseAlterConstraint(),
)
MySQL / MariaDB
Just make sure the connection string have the multistatement parameter set to true, and use:
testfixtures.New(
...
testfixtures.Dialect("mysql"), // or "mariadb"
)
Tested using the github.com/go-sql-driver/mysql driver.
ClickHouse
testfixtures.New(
...
testfixtures.Dialect("clickhouse"),
)
It’s possible generate values…
- id: {{sha256 “my-awesome-post}} title: My Awesome Post text: {{randomText}}
… or records
{{range \(post := \).Posts}}
- id: {{\(post.Id}} title: {{\)post.Title}} text: {{$post.Text}} {{end}}
## Generating fixtures for a existing database
The following code will generate a YAML file for each table of the database
into a given folder. It may be useful to boostrap a test scenario from a sample
database of your app.
```go
dumper, err := testfixtures.NewDumper(
testfixtures.DumpDatabase(db),
testfixtures.DumpDialect("postgres"), // or your database of choice
testfixtures.DumpDirectory("tmp/fixtures"),
testfixtures.DumpTables( // optional, will dump all table if not given
"posts",
"comments",
"tags",
),
)
if err != nil {
...
}
if err := dumper.Dump(); err != nil {
...
}
This was intended to run in small sample databases. It will likely break if run in a production/big database.
Gotchas
load
testfixtures -d postgres -c “postgres://user:password@localhost/database” -D testdata/fixtures
```bash
# dump
testfixtures --dump -d postgres -c "postgres://user:password@localhost/database" -D testdata/fixtures
The connection string changes for each database driver.
Use testfixtures --help
for all flags.
Alternatives
If you don’t think using fixtures is a good idea, you can try one of these packages instead:
- factory-go: Factory for Go. Inspired by Python’s Factory Boy and Ruby’s Factory Girl
- fixtory: go generate based type-safe, DRY, flexible test fixture factory
- go-txdb (Single transaction SQL driver for Go): Use a single database transaction for each functional test, so you can rollback to previous state between tests to have the same database state in all tests
- go-sqlmock: A mock for the sql.DB interface. This allow you to unit test database code without having to connect to a real database
- dbcleaner - Clean database for testing, inspired by database_cleaner for Ruby