Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch: impl Columns() []column.Interface method #1277

Merged
merged 3 commits into from
Jun 6, 2024
Merged

Batch: impl Columns() []column.Interface method #1277

merged 3 commits into from
Jun 6, 2024

Conversation

egsam98
Copy link
Contributor

@egsam98 egsam98 commented Apr 18, 2024

Hello, my proposal is to provide additional method Columns() []column.Interface for Batch interface. This might be useful when additional information about the table metadata is needed.

Example:

batch, err := chConn.PrepareBatch(ctx, fmt.Sprintf(`INSERT INTO %q`, table))
if err != nil {
        return errors.Wrap(err, "ClickHouse: Prepare batch")
}

structType, err := tableSchema(batch.Columns())
if err != nil {
        return err
}

func tableSchema(columns []column.Interface) (reflect.Type, error) {
	fields := make([]reflect.StructField, len(columns ))
	for i, col := range columns  {
		fieldType := col.ScanType()
		name := col.Name()
		fields[i] = reflect.StructField{
			Name: cases.Title(language.Und).String(name),
			Type: fieldType,
			Tag:  reflect.StructTag(fmt.Sprintf("json:%q ch:%q", name, name)),
		}
	}
	return reflect.StructOf(fields), nil
}

@CLAassistant
Copy link

CLAassistant commented Apr 18, 2024

CLA assistant check
All committers have signed the CLA.

@jkaflik
Copy link
Contributor

jkaflik commented Apr 25, 2024

Hi @egsam98

This might be useful when additional information about the table metadata is needed.

What additional data is needed? Perhaps we can try to expose more safe interface.

@egsam98
Copy link
Contributor Author

egsam98 commented Apr 26, 2024

I suppose the whole information provided by column.Interface might be useful for metaprogramming. As I pointed in example code, by means of Block() I can receive column info like name and Go type from proto.Block. I intentionally made this method returning shallow copy instead of pointer to somehow exclude possible modification. Unfortunately, there's no immutable pointers in Go.

As alternative solution I can replace Block() proto.Block with Columns() []column.Interface. If we speak about safer interface

Personally I use this approach to build dynamic struct via reflect.StructOf and then parse input JSON bytes. Also it would be useful to deal with impossibility to insert Go maps instead of structs:

  1. Obtain column names from proto.Block in the correct order
  2. Unwrap the map into slice by iteration for column names
  3. Use slice for Batch.Append() method.

@egsam98 egsam98 changed the title impl Batch.Block() method Batch: impl Columns() []column.Interface method Apr 30, 2024
@egsam98
Copy link
Contributor Author

egsam98 commented Apr 30, 2024

Replaced with Columns []column.Interface method

@jkaflik
Copy link
Contributor

jkaflik commented Jun 3, 2024

@egsam98 Columns method works for me. Please add a test case that does a simple assertion.

@egsam98
Copy link
Contributor Author

egsam98 commented Jun 4, 2024

@jkaflik Hi, let me know if I've added test case in a wrong place of codebase. Thank you

@jkaflik jkaflik merged commit 8ad6ec6 into ClickHouse:main Jun 6, 2024
10 checks passed
codeboten referenced this pull request in open-telemetry/opentelemetry-collector-contrib Jul 10, 2024
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
|
[github.com/ClickHouse/clickhouse-go/v2](https://github.com/ClickHouse/clickhouse-go)
| `v2.23.0` -> `v2.26.0` |
[![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fClickHouse%2fclickhouse-go%2fv2/v2.26.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/go/github.com%2fClickHouse%2fclickhouse-go%2fv2/v2.26.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/go/github.com%2fClickHouse%2fclickhouse-go%2fv2/v2.23.0/v2.26.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fClickHouse%2fclickhouse-go%2fv2/v2.23.0/v2.26.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|

---

> [!WARNING]
> Some dependencies could not be looked up. Check the Dependency
Dashboard for more information.

---

### Release Notes

<details>
<summary>ClickHouse/clickhouse-go
(github.com/ClickHouse/clickhouse-go/v2)</summary>

###
[`v2.26.0`](https://github.com/ClickHouse/clickhouse-go/blob/HEAD/CHANGELOG.md#v2260-2024-06-25----Release-notes-generated-using-configuration-in-githubreleaseyml-at-main---)

[Compare
Source](https://github.com/ClickHouse/clickhouse-go/compare/v2.25.0...v2.26.0)

#### What's Changed

##### Enhancements 🎉

- Reintroduce the random connection strategy by
[@&#8203;larry-cdn77](https://github.com/larry-cdn77) in
[https://github.com/ClickHouse/clickhouse-go/pull/1313](https://github.com/ClickHouse/clickhouse-go/pull/1313)
- Make custom debug log function on-par with the built-in one by
[@&#8203;vespian](https://github.com/vespian) in
[https://github.com/ClickHouse/clickhouse-go/pull/1317](https://github.com/ClickHouse/clickhouse-go/pull/1317)
- Remove date overflow check as it's normalised by ClickHouse server by
[@&#8203;gogingersnap777](https://github.com/gogingersnap777) in
[https://github.com/ClickHouse/clickhouse-go/pull/1315](https://github.com/ClickHouse/clickhouse-go/pull/1315)
- Batch: impl `Columns() []column.Interface` method by
[@&#8203;egsam98](https://github.com/egsam98) in
[https://github.com/ClickHouse/clickhouse-go/pull/1277](https://github.com/ClickHouse/clickhouse-go/pull/1277)

##### Fixes 🐛

- Fix rows.Close do not return too early by
[@&#8203;yujiarista](https://github.com/yujiarista) in
[https://github.com/ClickHouse/clickhouse-go/pull/1314](https://github.com/ClickHouse/clickhouse-go/pull/1314)
- Setting `X-Clickhouse-SSL-Certificate-Auth` header correctly given
`X-ClickHouse-Key` by
[@&#8203;gogingersnap777](https://github.com/gogingersnap777) in
[https://github.com/ClickHouse/clickhouse-go/pull/1316](https://github.com/ClickHouse/clickhouse-go/pull/1316)
- Retry on network errors and fix retries on async inserts with
`database/sql` interface by
[@&#8203;tommyzli](https://github.com/tommyzli) in
[https://github.com/ClickHouse/clickhouse-go/pull/1330](https://github.com/ClickHouse/clickhouse-go/pull/1330)
- BatchInsert parentheses issue fix by
[@&#8203;ramzes642](https://github.com/ramzes642) in
[https://github.com/ClickHouse/clickhouse-go/pull/1327](https://github.com/ClickHouse/clickhouse-go/pull/1327)

##### Other Changes 🛠

- ClickHouse 24.5 by [@&#8203;jkaflik](https://github.com/jkaflik) in
[https://github.com/ClickHouse/clickhouse-go/pull/1319](https://github.com/ClickHouse/clickhouse-go/pull/1319)
- Align `allow_suspicious_low_cardinality_types` and `
allow_suspicious_low_cardinality_types ` settings in tests due to
ClickHouse Cloud incompatibility by
[@&#8203;jkaflik](https://github.com/jkaflik) in
[https://github.com/ClickHouse/clickhouse-go/pull/1331](https://github.com/ClickHouse/clickhouse-go/pull/1331)
- Use HTTPs scheme in std connection failover tests by
[@&#8203;jkaflik](https://github.com/jkaflik) in
[https://github.com/ClickHouse/clickhouse-go/pull/1332](https://github.com/ClickHouse/clickhouse-go/pull/1332)

#### New Contributors

- [@&#8203;larry-cdn77](https://github.com/larry-cdn77) made their
first contribution in
[https://github.com/ClickHouse/clickhouse-go/pull/1313](https://github.com/ClickHouse/clickhouse-go/pull/1313)
- [@&#8203;vespian](https://github.com/vespian) made their first
contribution in
[https://github.com/ClickHouse/clickhouse-go/pull/1317](https://github.com/ClickHouse/clickhouse-go/pull/1317)
- [@&#8203;gogingersnap777](https://github.com/gogingersnap777) made
their first contribution in
[https://github.com/ClickHouse/clickhouse-go/pull/1315](https://github.com/ClickHouse/clickhouse-go/pull/1315)
- [@&#8203;yujiarista](https://github.com/yujiarista) made their first
contribution in
[https://github.com/ClickHouse/clickhouse-go/pull/1314](https://github.com/ClickHouse/clickhouse-go/pull/1314)
- [@&#8203;egsam98](https://github.com/egsam98) made their first
contribution in
[https://github.com/ClickHouse/clickhouse-go/pull/1277](https://github.com/ClickHouse/clickhouse-go/pull/1277)
- [@&#8203;tommyzli](https://github.com/tommyzli) made their first
contribution in
[https://github.com/ClickHouse/clickhouse-go/pull/1330](https://github.com/ClickHouse/clickhouse-go/pull/1330)
- [@&#8203;ramzes642](https://github.com/ramzes642) made their first
contribution in
[https://github.com/ClickHouse/clickhouse-go/pull/1327](https://github.com/ClickHouse/clickhouse-go/pull/1327)

**Full Changelog**:
ClickHouse/clickhouse-go@v2.25.0...v2.26.0

###
[`v2.25.0`](https://github.com/ClickHouse/clickhouse-go/blob/HEAD/CHANGELOG.md#v2250-2024-05-28----Release-notes-generated-using-configuration-in-githubreleaseyml-at-main---)

[Compare
Source](https://github.com/ClickHouse/clickhouse-go/compare/v2.24.0...v2.25.0)

#### What's Changed

##### Fixes 🐛

- Add a compatibility layer for a database/sql driver to work with
sql.NullString and ClickHouse nullable column by
[@&#8203;jkaflik](https://github.com/jkaflik) in
[https://github.com/ClickHouse/clickhouse-go/pull/1306](https://github.com/ClickHouse/clickhouse-go/pull/1306)

##### Other Changes 🛠

- Use Go 1.22 in head tests by
[@&#8203;jkaflik](https://github.com/jkaflik) in
[https://github.com/ClickHouse/clickhouse-go/pull/1305](https://github.com/ClickHouse/clickhouse-go/pull/1305)
- Skip flaky 1127 test by
[@&#8203;jkaflik](https://github.com/jkaflik) in
[https://github.com/ClickHouse/clickhouse-go/pull/1307](https://github.com/ClickHouse/clickhouse-go/pull/1307)

**Full Changelog**:
ClickHouse/clickhouse-go@v2.24.0...v2.25.0

###
[`v2.24.0`](https://github.com/ClickHouse/clickhouse-go/blob/HEAD/CHANGELOG.md#v2240-2024-05-08----Release-notes-generated-using-configuration-in-githubreleaseyml-at-main---)

[Compare
Source](https://github.com/ClickHouse/clickhouse-go/compare/v2.23.2...v2.24.0)

#### What's Changed

##### Enhancements 🎉

- Always compress responses when the client compression is on by
[@&#8203;zhkvia](https://github.com/zhkvia) in
[https://github.com/ClickHouse/clickhouse-go/pull/1286](https://github.com/ClickHouse/clickhouse-go/pull/1286)
- Optional flag to close query with flush by
[@&#8203;hongker](https://github.com/hongker) in
[https://github.com/ClickHouse/clickhouse-go/pull/1276](https://github.com/ClickHouse/clickhouse-go/pull/1276)

##### Fixes 🐛

- Fix prepare batch does not break on `values` substring in table name
by [@&#8203;Wang](https://github.com/Wang) in
[https://github.com/ClickHouse/clickhouse-go/pull/1290](https://github.com/ClickHouse/clickhouse-go/pull/1290)
- Fix nil checks when appending slices of pointers by
[@&#8203;markandrus](https://github.com/markandrus) in
[https://github.com/ClickHouse/clickhouse-go/pull/1283](https://github.com/ClickHouse/clickhouse-go/pull/1283)

##### Other Changes 🛠

- Don't recreate keys from LC columns from direct stream by
[@&#8203;genzgd](https://github.com/genzgd) in
[https://github.com/ClickHouse/clickhouse-go/pull/1291](https://github.com/ClickHouse/clickhouse-go/pull/1291)

#### New Contributors

- [@&#8203;zhkvia](https://github.com/zhkvia) made their first
contribution in
[https://github.com/ClickHouse/clickhouse-go/pull/1286](https://github.com/ClickHouse/clickhouse-go/pull/1286)

**Full Changelog**:
ClickHouse/clickhouse-go@v2.23.2...v2.24.0

###
[`v2.23.2`](https://github.com/ClickHouse/clickhouse-go/blob/HEAD/CHANGELOG.md#v2232-2024-04-25----Release-notes-generated-using-configuration-in-githubreleaseyml-at-main---)

[Compare
Source](https://github.com/ClickHouse/clickhouse-go/compare/v2.23.1...v2.23.2)

#### What's Changed

##### Fixes 🐛

- Fixed panic on concurrent context key map write by
[@&#8203;Wang](https://github.com/Wang) in
[https://github.com/ClickHouse/clickhouse-go/pull/1284](https://github.com/ClickHouse/clickhouse-go/pull/1284)

##### Other Changes 🛠

- Fix ClickHouse Terraform provider version by
[@&#8203;jkaflik](https://github.com/jkaflik) in
[https://github.com/ClickHouse/clickhouse-go/pull/1285](https://github.com/ClickHouse/clickhouse-go/pull/1285)

#### New Contributors

- [@&#8203;Wang](https://github.com/Wang) made their first
contribution in
[https://github.com/ClickHouse/clickhouse-go/pull/1284](https://github.com/ClickHouse/clickhouse-go/pull/1284)

**Full Changelog**:
ClickHouse/clickhouse-go@v2.23.1...v2.23.2

###
[`v2.23.1`](https://github.com/ClickHouse/clickhouse-go/blob/HEAD/CHANGELOG.md#v2231-2024-04-15----Release-notes-generated-using-configuration-in-githubreleaseyml-at-main---)

[Compare
Source](https://github.com/ClickHouse/clickhouse-go/compare/v2.23.0...v2.23.1)

#### What's Changed

##### Fixes 🐛

- Zero-value timestamp to be formatted as toDateTime(0) in bind by
[@&#8203;jkaflik](https://github.com/jkaflik) in
[https://github.com/ClickHouse/clickhouse-go/pull/1260](https://github.com/ClickHouse/clickhouse-go/pull/1260)

##### Other Changes 🛠

- Update
[#&#8203;1127](https://github.com/ClickHouse/clickhouse-go/issues/1127)
test case to reproduce a progress handle when exception is thrown by
[@&#8203;jkaflik](https://github.com/jkaflik) in
[https://github.com/ClickHouse/clickhouse-go/pull/1259](https://github.com/ClickHouse/clickhouse-go/pull/1259)
- Set max parallel for GH jobs by
[@&#8203;jkaflik](https://github.com/jkaflik) in
[https://github.com/ClickHouse/clickhouse-go/pull/1261](https://github.com/ClickHouse/clickhouse-go/pull/1261)
- Ensure test container termination by
[@&#8203;jkaflik](https://github.com/jkaflik) in
[https://github.com/ClickHouse/clickhouse-go/pull/1274](https://github.com/ClickHouse/clickhouse-go/pull/1274)

**Full Changelog**:
ClickHouse/clickhouse-go@v2.23.0...v2.23.1

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any
time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector-contrib).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MTMuMiIsInVwZGF0ZWRJblZlciI6IjM3LjQyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiLCJyZW5vdmF0ZWJvdCJdfQ==-->

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: opentelemetrybot <[email protected]>
Co-authored-by: Yang Song <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants