Skip to content

Commit

Permalink
docs - database troubles (metabase#25591)
Browse files Browse the repository at this point in the history
  • Loading branch information
nllho authored Sep 23, 2022
1 parent 2067cd9 commit 416146c
Show file tree
Hide file tree
Showing 12 changed files with 145 additions and 111 deletions.
2 changes: 1 addition & 1 deletion docs/databases/connecting.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@ Go to **Admin settings** > **Databases** > your database and click **Remove this

## Troubleshooting

- [Troubleshooting database connections](../troubleshooting-guide/datawarehouse.md)
- [Troubleshooting database connections](../troubleshooting-guide/db-connection.md)
- [Troubleshooting syncs, scans, and fingerprinting](../troubleshooting-guide/sync-fingerprint-scan.md)
- Search or ask the [Metabase community](https://discourse.metabase.com/).
- Search for [known bugs or limitations](../troubleshooting-guide/known-issues.md).
Expand Down
2 changes: 1 addition & 1 deletion docs/databases/connections/bigquery.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ FROM [my_dataset.my_table]

## Troubleshooting

If you're having trouble with your BigQuery connection, you can check out this [troubleshooting guide](../../troubleshooting-guide/bigquery-drive.md) that covers BigQuery issues, [this one](../../troubleshooting-guide/datawarehouse.md) on data warehouse connections, or visit [Metabase's discussion forum](https://discourse.metabase.com/search?q=bigquery) to see if someone has encountered and resolved a similar issue.
If you're having trouble with your BigQuery connection, you can check out this [troubleshooting guide](../../troubleshooting-guide/bigquery-drive.md) that covers BigQuery issues, [this one](../../troubleshooting-guide/db-connection.md) on data warehouse connections, or visit [Metabase's discussion forum](https://discourse.metabase.com/search?q=bigquery) to see if someone has encountered and resolved a similar issue.

## Further reading

Expand Down
4 changes: 2 additions & 2 deletions docs/databases/connections/mongodb.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ This article covers:

## How Metabase syncs data in MongoDB

Because MongoDB contains unstructured data, Metabase takes a different approach to syncing your database's metadata. To get a sense of the schema, Metabase will scan the first ten thousand documents of each collection in your MongoDB. This sampling helps Metabase do things like differentiate datetime fields from string fields, and provide people with pre-populated filters. The reason Metabase only scans a sample of the documents is because scanning every document in every collection on every sync would be put too much strain on your database. And while the sampling does a pretty good job keeping Metabase up to date, it can also mean that new fields can sometimes fall through the cracks, leading to visualization issues, or even fields failing to appear in your results. For more info, check out our [troubleshooting guide](../../troubleshooting-guide/datawarehouse.md).
Because MongoDB contains unstructured data, Metabase takes a different approach to syncing your database's metadata. To get a sense of the schema, Metabase will scan the first ten thousand documents of each collection in your MongoDB. This sampling helps Metabase do things like differentiate datetime fields from string fields, and provide people with pre-populated filters. The reason Metabase only scans a sample of the documents is because scanning every document in every collection on every sync would be put too much strain on your database. And while the sampling does a pretty good job keeping Metabase up to date, it can also mean that new fields can sometimes fall through the cracks, leading to visualization issues, or even fields failing to appear in your results. For more info, check out our [troubleshooting guide](../../troubleshooting-guide/db-connection.md).

## Connecting to MongoDB

Expand Down Expand Up @@ -117,4 +117,4 @@ One workaround is to include all possible keys in the first document of the coll

## Further reading

See our troubleshooting guide for [troubleshooting your connection](../../troubleshooting-guide/datawarehouse.md).
See our troubleshooting guide for [troubleshooting your connection](../../troubleshooting-guide/db-connection.md).
2 changes: 1 addition & 1 deletion docs/troubleshooting-guide/bigquery-drive.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,4 @@ You may have forgotten to [share your Google Drive source](../databases/connecti

## Further reading

- [Troubleshooting database connections](./datawarehouse.md)
- [Troubleshooting database connections](./db-connection.md)
2 changes: 1 addition & 1 deletion docs/troubleshooting-guide/cant-see-tables.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ If you can't connect to the database with another application, the problem is pr

**Steps to take:**

If your connection isn't working, go to [Troubleshooting database connections](./datawarehouse.md).
If your connection isn't working, go to [Troubleshooting database connections](./db-connection.md).

## Does the table exist?

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
---
title: Troubleshooting database connections
redirect_from:
- /docs/latest/troubleshooting-guide/datawarehouse
---

# Troubleshooting database connections
Expand Down Expand Up @@ -95,6 +97,11 @@ psql -h HOSTNAME -p PORT -d DATABASENAME -U DATABASEUSER
SELECT 1
```
## Related problems
- [My connection or query is timing out](./timeout.md).
- [My database is slow](./db-performance.md).
## Are you still stuck?
If you can’t solve your problem using the troubleshooting guides:
Expand Down
83 changes: 83 additions & 0 deletions docs/troubleshooting-guide/db-performance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
---
title: Troubleshooting database performance
---

# Troubleshooting database performance

This guide deals with databases or data warehouses that are [connected to Metabase](../databases/connecting.md) as data sources.

To fix problems with your Metabase [application database](../installation-and-operation/configuring-application-database.md), check out these troubleshooting guides:

- [Running Metabase](./running.md).
- [Running Metabase on Docker](./docker.md).
- [Using or migrating from an H2 application database](./loading-from-h2.md).

## Identifying bottlenecks

1. Optional: use Metabase's [auditing tools](../usage-and-performance-tools/audit.md) to look at your Metabase usage stats.\*
2. Go to your database's server logs and check whether:
- Your tables are growing in size,
- More people are using Metabase to access your database,
- People are accessing your database more often, or
- A script or application (other than Metabase) is accessing the database frequently.
3. If specific tables are being queried a lot, try [Optimizing your table schemas](https://www.metabase.com/learn/administration/making-dashboards-faster#organize-data-to-anticipate-common-questions).
4. Run a question from Metabase, then [run the same query](../questions/query-builder/introduction.md#viewing-the-sql-that-powers-your-question) directly against your database.
- If the queries take about the same time, your data or usage might be outgrowing your database. You can give your database more resources, or consider [upgrading your hardware](https://www.metabase.com/learn/analytics/which-data-warehouse).
- If the query in Metabase takes longer than a direct query against your database, you might need to adjust the deployment of your Metabase app. Check out some options in [Metabase at scale](https://www.metabase.com/learn/administration/metabase-at-scale).
5. If a script or third-party application is hitting your database with a lot of queries at a time:
- Stop your script or application, and [clear any queued queries](#clearing-queued-queries).
- Recommended: add a timeout to your script, schedule the script or application to run during off-hours, or replicate your database (and point your tools there instead).

\* Available on paid plans.

## Resetting a database connection

1. Go to **Settings** > **Admin settings** > **Databases** > your database.
2. Click **Save changes** (without making changes) to reset Metabase's connections to your database.
3. Alternatively: kill the connection(s) directly from your database.

**Explanation**

"Turn it off, and on again" by disconnecting and reconnecting your database---an easy sanity check that can save you a lot of time.

In general, Metabase will try to close hanging connections to your database after 10 minutes, and then again after 20 minutes. But if your database doesn't respond, you may need to close the connection to Metabase from the database side.

## Clearing queued queries

1. Stop the process (e.g., a script, or a dashboard with [too many cards](./my-dashboard-is-slow.md#dashboard-has-over-10-cards) that's launching a lot of queries at once.
2. Go to your database server and stop all queries (from Metabase) that are in progress.
3. Optional: Increase the [number of connections to your database](../configuring-metabase/environment-variables.md#mb_jdbc_data_warehouse_max_connection_pool_size).

**Explanation**

If someone or something creates 100 queries at the same time, this stampede of queries will take up all of the available connections between Metabase and your database, preventing any new queries from running. If other people continue running questions and dashboards while the first 100 queries are still in progress, the queue will grow at a faster rate than your database can keep up with.

## Managing resource-intensive queries

1. [Reschedule or disable Metabase syncs and scans](../databases/connecting.md#syncing-and-scanning-databases).

**Explanation**

By default, Metabase makes regular sync and scan queries against your database to keep your tables up to date, get fresh values for filter dropdowns, and make helpful suggestions. If you've got a very large database, you can choose to trigger these queries manually instead of on a schedule.

## Questions that use number, date, or timestamp columns

1. Update your database schema so that the columns are typed correctly.
2. [Scan the updated columns](../databases/connecting.md#manually-scanning-column-values) to bring the changes into Metabase.

**Explanation**

If a question uses data stored as the wrong [data type](https://www.metabase.com/learn/databases/data-types-overview) in your database (most common with number, date, or timestamp values stored as strings), Metabase will generate a query that asks your database to convert the values on the fly. Typing your columns correctly at the schema level will help your database avoid that extra step to return results faster in Metabase.

## Related problems

- [My connection or query is timing out](./timeout.md).
- [I can't connect to a database](./db-connection.md).
- [My dashboard is slow or failing to load](./my-dashboard-is-slow.md).

## Are you still stuck?

If you can’t solve your problem using the troubleshooting guides:

- Search or ask the [Metabase community](https://discourse.metabase.com/).
- Search for [known bugs or limitations](./known-issues.md).
1 change: 1 addition & 0 deletions docs/troubleshooting-guide/error-message.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ title: Troubleshooting error messages
### Common Metabase error messages

- [Your question took too long](./timeout.md).
- [Still waiting...](./my-dashboard-is-slow.md).

[api-error-message]: #api-error-messages
[debugging-sql-syntax]: https://www.metabase.com/learn/debugging-sql/sql-syntax.html
Expand Down
6 changes: 4 additions & 2 deletions docs/troubleshooting-guide/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,10 @@ Problems, their causes, how to detect them, and how to fix them.

## Databases

- [I can't connect to a database][datawarehouse].
- [I can't connect to a database][db-connection].
- [I can't see my tables][cant-see-tables].
- [The data in Metabase doesn't match my database][sync-fingerprint-scan].
- [My database is slow][db-performance].
- [My connection or query is timing out][timeout].

## Questions and dashboards
Expand Down Expand Up @@ -92,7 +93,8 @@ Metabase adds new features and squashes bugs with each release. [Upgrading to th
[cant-see-tables]: ./cant-see-tables.md
[chrome]: https://developers.google.com/web/tools/chrome-devtools/open#console
[cloud]: https://www.metabase.com/start/
[datawarehouse]: ./datawarehouse.md
[db-connection]: ./db-connection.md
[db-performance]: ./db-performance.md
[docker]: ./docker.md
[edge]: https://docs.microsoft.com/en-us/microsoft-edge/devtools-guide-chromium
[error-message]: error-message.md
Expand Down
Loading

0 comments on commit 416146c

Please sign in to comment.