Skip to content

Release-15: Cherry pick vtorc no cgo#12223

Merged
frouioui merged 3 commits intovitessio:release-15.0from
dbussink:cherry-pick-vtorc-no-cgo
Feb 6, 2023
Merged

Release-15: Cherry pick vtorc no cgo#12223
frouioui merged 3 commits intovitessio:release-15.0from
dbussink:cherry-pick-vtorc-no-cgo

Conversation

@dbussink
Copy link
Member

@dbussink dbussink commented Feb 2, 2023

Manual backport of #12214

Checklist

  • "Backport to:" labels have been added if this change should be back-ported
  • Tests were added or are not required
  • Documentation was added or is not required

@dbussink
Move vtorc from go-sqlite3 to modernc.org/sqlite (vitessio#12214)
2305083 
* Move vtorc from go-sqlite3 to modernc.org/sqlite

This moves vtorc from the go-sqlite3 library that uses CGO, to use
modernc.org/sqlite which is a pure Go implementation.

vtorc is the only component we have to build with CGO but it's causing
pain for releases since we need to build it against an old Linux for
linking against glibc.

Using modernc.org/sqlite allows for using Go only again and makes all
Vitess components buildable without CGO.

In
https://datastation.multiprocess.io/blog/2022-05-12-sqlite-in-go-with-and-without-cgo.html
someone ran some basic benchmarks. It shows that the pure Go version can
be twice as slow, but the usage of vtorc is very limited and we operate
on small datasets, so I think the performance impact purely of a
somewhat slower sqlite implementation is negligable.

None of this is in a hot query serving path or anything like that, so I
have little concern performance wise.

Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>

* Fix error handling in RowToArray

Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>

---------

Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
@dbussink dbussink added Type: Enhancement Logical improvement (somewhere between a bug and feature) Backport This is a backport Component: VTOrc Vitess Orchestrator integration labels Feb 2, 2023
@vitess-bot vitess-bot bot added NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsWebsiteDocsUpdate What it says labels Feb 2, 2023
@vitess-bot
Copy link
Contributor

vitess-bot bot commented Feb 2, 2023 •
edited by frouioui
Loading

Review Checklist

Hello reviewers! 👋 Please follow this checklist when reviewing this Pull Request.

General

  • Ensure that the Pull Request has a descriptive title.
  • If this is a change that users need to know about, please apply the release notes (needs details) label so that merging is blocked unless the summary release notes document is included.
  • If a test is added or modified, there should be a documentation on top of the test to explain what the expected behavior is what the test does.

If a new flag is being introduced:

  • Is it really necessary to add this flag?
  • Flag names should be clear and intuitive (as far as possible)
  • Help text should be descriptive.
  • Flag names should use dashes (-) as word separators rather than underscores (_).

If a workflow is added or modified:

  • Each item in Jobs should be named in order to mark it as required.
  • If the workflow should be required, the maintainer team should be notified.

Bug fixes

  • There should be at least one unit or end-to-end test.
  • The Pull Request description should include a link to an issue that describes the bug.

Non-trivial changes

  • There should be some code comments as to why things are implemented the way they are.

New/Existing features

  • Should be documented, either by modifying the existing documentation or creating new documentation.
  • New features should have a link to a feature request issue or an RFC that documents the use cases, corner cases and test cases.

Backward compatibility

  • Protobuf changes should be wire-compatible.
  • Changes to _vt tables and RPCs need to be backward compatible.
  • vtctl command output order should be stable and awk-able.
  • RPC changes should be compatible with vitess-operator
  • If a flag is removed, then it should also be removed from VTop, if used there.

@dbussink dbussink mentioned this pull request Feb 2, 2023
Merged
3 tasks
GuptaManan100 and others added 2 commits February 3, 2023 03:22
@GuptaManan100
empty-commit
6fc2a30 
Signed-off-by: Manan Gupta <manan@planetscale.com>
@dbussink
Run go mod tidy
9b88cee 
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
@dbussink dbussink removed NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsWebsiteDocsUpdate What it says labels Feb 6, 2023
@frouioui frouioui merged commit c1d8c14 into vitessio:release-15.0 Feb 6, 2023
@dbussink dbussink deleted the cherry-pick-vtorc-no-cgo branch February 6, 2023 09:27
@the-maldridge
Copy link

the-maldridge commented Feb 8, 2023

Is there a handy crystal ball that I can look at for when the next point release of the release-15.0 branch is expected? This doesn't appear to be part of a milestone or I would have checked there.

@austenLacy austenLacy mentioned this pull request Apr 26, 2023
Merged
@austenLacy austenLacy mentioned this pull request Aug 1, 2023
Closed
timvaillancourt pushed a commit to slackhq/vitess that referenced this pull request Aug 14, 2023
@dbussink @GuptaManan100 @timvaillancourt
Release-15: Cherry pick vtorc no cgo (vitessio#12223)
8a297ff 
* Move vtorc from go-sqlite3 to modernc.org/sqlite (vitessio#12214)

* Move vtorc from go-sqlite3 to modernc.org/sqlite

This moves vtorc from the go-sqlite3 library that uses CGO, to use
modernc.org/sqlite which is a pure Go implementation.

vtorc is the only component we have to build with CGO but it's causing
pain for releases since we need to build it against an old Linux for
linking against glibc.

Using modernc.org/sqlite allows for using Go only again and makes all
Vitess components buildable without CGO.

In
https://datastation.multiprocess.io/blog/2022-05-12-sqlite-in-go-with-and-without-cgo.html
someone ran some basic benchmarks. It shows that the pure Go version can
be twice as slow, but the usage of vtorc is very limited and we operate
on small datasets, so I think the performance impact purely of a
somewhat slower sqlite implementation is negligable.

None of this is in a hot query serving path or anything like that, so I
have little concern performance wise.

Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>

* Fix error handling in RowToArray

Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>

---------

Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>

* empty-commit

Signed-off-by: Manan Gupta <manan@planetscale.com>

* Run go mod tidy

Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>

---------

Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
Co-authored-by: Manan Gupta <manan@planetscale.com>
@timvaillancourt timvaillancourt mentioned this pull request Aug 14, 2023
Merged
4 tasks
timvaillancourt added a commit to slackhq/vitess that referenced this pull request Aug 15, 2023
@timvaillancourt @dbussink @GuptaManan100
slack-vitess-r14.0.5: backport vitessio#12223 to fix vtorc GLIBC …
772645c 
…error (#117)

* Release-15: Cherry pick vtorc no cgo (vitessio#12223)

* Move vtorc from go-sqlite3 to modernc.org/sqlite (vitessio#12214)

* Move vtorc from go-sqlite3 to modernc.org/sqlite

This moves vtorc from the go-sqlite3 library that uses CGO, to use
modernc.org/sqlite which is a pure Go implementation.

vtorc is the only component we have to build with CGO but it's causing
pain for releases since we need to build it against an old Linux for
linking against glibc.

Using modernc.org/sqlite allows for using Go only again and makes all
Vitess components buildable without CGO.

In
https://datastation.multiprocess.io/blog/2022-05-12-sqlite-in-go-with-and-without-cgo.html
someone ran some basic benchmarks. It shows that the pure Go version can
be twice as slow, but the usage of vtorc is very limited and we operate
on small datasets, so I think the performance impact purely of a
somewhat slower sqlite implementation is negligable.

None of this is in a hot query serving path or anything like that, so I
have little concern performance wise.

Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>

* Fix error handling in RowToArray

Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>

---------

Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>

* empty-commit

Signed-off-by: Manan Gupta <manan@planetscale.com>

* Run go mod tidy

Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>

---------

Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
Co-authored-by: Manan Gupta <manan@planetscale.com>

* Fix Makefile

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

* Undo dir rename

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

---------

Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>
Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com>
Co-authored-by: Manan Gupta <manan@planetscale.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@frouioui frouioui frouioui approved these changes

@deepthi deepthi Awaiting requested review from deepthi

@shlomi-noach shlomi-noach Awaiting requested review from shlomi-noach shlomi-noach is a code owner

@rsajwani rsajwani Awaiting requested review from rsajwani rsajwani is a code owner

@ajm188 ajm188 Awaiting requested review from ajm188

+1 more reviewer

@GuptaManan100 GuptaManan100 GuptaManan100 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

Backport This is a backport Component: VTOrc Vitess Orchestrator integration Type: Enhancement Logical improvement (somewhere between a bug and feature)

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@dbussink @the-maldridge @frouioui @GuptaManan100