Skip to content

Decoding command output in Windows#2084

Merged
eeeebbbbrrrr merged 3 commits intopgcentralfoundation:developfrom
if0ne:windows-utf8
Jun 28, 2025
Merged

Decoding command output in Windows#2084
eeeebbbbrrrr merged 3 commits intopgcentralfoundation:developfrom
if0ne:windows-utf8

Conversation

@if0ne
Copy link
Contributor

@if0ne if0ne commented Jun 11, 2025

Closes #2069

@usamoi
Copy link
Contributor

usamoi commented Jun 15, 2025

As I understand it, using from_utf8_lossy is not suitable for double-byte encodings. A better approach would be to use GetACP to retrieve the system code page and then use the corresponding encoding from encoding_rs to convert it to UTF-8.

@if0ne
Copy link
Contributor Author

if0ne commented Jun 16, 2025

from_utf8_lossy will help avoid crashes from panic. The fact that REPLACEMENT_CHARACTERs will be displayed is a different problem imo.

I could solve the encoding issue with a third-party crate, but I have little experience with open-source and don't know if it's ok to add an additional dependency without any discussion from the major contributors.

@YohDeadfall
Copy link
Contributor

YohDeadfall commented Jun 16, 2025

It can be a Windows only dependency then as it requires special handling anyway, and pgrx already has some Windows specific fixes. So, why not?

@if0ne
Copy link
Contributor Author

if0ne commented Jun 17, 2025

Ok, I'll fix the Windows encoding problem then.

@if0ne if0ne marked this pull request as draft June 17, 2025 07:23
@if0ne if0ne changed the title Replaced from_utf8 with from_utf8_lossy for command outputs Decoding command output in Windows Jun 17, 2025
@if0ne if0ne marked this pull request as ready for review June 18, 2025 20:19
@eeeebbbbrrrr eeeebbbbrrrr merged commit 2c1c1b8 into pgcentralfoundation:develop Jun 28, 2025
@eeeebbbbrrrr eeeebbbbrrrr mentioned this pull request Jun 28, 2025
Merged
eeeebbbbrrrr added a commit that referenced this pull request Jun 28, 2025
@eeeebbbbrrrr
Update version to 0.15.0 (#2096)
aa1c67c 
Welcome to pgrx v0.15.0. This begins a new series for pgrx that includes
support for Postgres 18. As of this release, that means Postgres
18beta1.

This release does contain a few breaking API changes but they're largely
mechanical. Don't worry, the compiler will let you know!

As always, please install our CI tool with `cargo install cargo-pgrx
--version 0.15.0 --locked` and then run `cargo pgrx upgrade` in all of
your extension crates.

If you want to start working with Postgres 18beta1, you'll also need to
re-init your pgrx environment with `cargo pgrx init`. That will
automatically detect all the latest Postgres versions, including
18beta1.

At the top here, I'd like to thank @silver-ymz for the 18beta1 support.
It was a pleasant surprise to see that work come from the community --
it's no easy task to add a new Postgres version to pgrx!

That said, as Postgres 18 is currently beta, you should consider pgrx'
support for it as beta too. Please report any problems with 18beta1 (or
discrepancies with other versions) as GitHub issues.

Also, this release requires rust v1.88.0 or greater. `if-let` chains are
now a thing and we're not afraid to use them.

# What's Changed

## Postgres 18beta1 Support

* Support Postgres 18beta1 by @silver-ymz in
#2056
* pg18 support: add header and implement `#define` by @eeeebbbbrrrr in
#2094
* improve pg_magic_func by @usamoi in
#2088


## More Headers

* Added `catalog/heap.h` binding by @ccleve in
#2072
* include `utils/pg_status.h` by @eeeebbbbrrrr in
#2091


## `cargo-pgrx` improvements

* Pass `LLVM_*` variables to `--runas` command by @theory in
#2083
* `does_db_exist()`: fix `psql` argument order by @eeeebbbbrrrr in
#2093
* `cargo pgrx regress` output is no longer fully buffered by
@eeeebbbbrrrr in #2095
* Detect `pgrx_embed` name from lib name by @YohDeadfall in
#2035
* Fixed error message if no artifact found by @YohDeadfall in
#2034
* `cargo-pgrx`: use system certificate store for HTTPS validation by
@charmitro in #2074
* Decoding command output in Windows by @if0ne in
#2084


## Breaking Changes

* fix GUC by @usamoi in
#2064
* refactor GUC by @usamoi in
#2066

## New Stuff

* Added `pg_binary_protocol` attribute to derive send and receive
functions for `PostgresType` by @LucaCappelletti94 in
#2068
* Expose guc hooks by @thesuhas in
#2075
* Allows to create multiple aggregates for the same Rust type by @if0ne
in #2078



## General Code Cleanup

* `cargo clippy --fix` by @eeeebbbbrrrr in
#2092
* Use `if-let` to unpack Options by @stuhood in
#2089
* docs: fix typo in `rust_byte_slice_to_bytea()` docs by @burmecia in
#2071
* Added a missing `#[doc(hidden)]` by @LucaCappelletti94 in
#2079

## Administrative

* Updated Fedora to latest in CI by @YohDeadfall in
#2085
* fix ci on beta rust (1.89) by @usamoi in
#2087

## New Contributors

Much thanks to our new contributors! Your work is sincerely appreciated!

* @charmitro made their first contribution in
#2074
* @thesuhas made their first contribution in
#2075
* @if0ne made their first contribution in
#2084
* @stuhood made their first contribution in
#2089

**Full Changelog**:
v0.14.3...v0.15.0
@BrewTestBot BrewTestBot mentioned this pull request Jun 28, 2025
Merged
daamien pushed a commit to daamien/pgrx that referenced this pull request Dec 15, 2025
@if0ne @daamien
Decoding command output in Windows (pgcentralfoundation#2084)
5fa6fa7 
Closes pgcentralfoundation#2069
daamien pushed a commit to daamien/pgrx that referenced this pull request Dec 15, 2025
@eeeebbbbrrrr @daamien
Update version to 0.15.0 (pgcentralfoundation#2096)
f327a99 
Welcome to pgrx v0.15.0. This begins a new series for pgrx that includes
support for Postgres 18. As of this release, that means Postgres
18beta1.

This release does contain a few breaking API changes but they're largely
mechanical. Don't worry, the compiler will let you know!

As always, please install our CI tool with `cargo install cargo-pgrx
--version 0.15.0 --locked` and then run `cargo pgrx upgrade` in all of
your extension crates.

If you want to start working with Postgres 18beta1, you'll also need to
re-init your pgrx environment with `cargo pgrx init`. That will
automatically detect all the latest Postgres versions, including
18beta1.

At the top here, I'd like to thank @silver-ymz for the 18beta1 support.
It was a pleasant surprise to see that work come from the community --
it's no easy task to add a new Postgres version to pgrx!

That said, as Postgres 18 is currently beta, you should consider pgrx'
support for it as beta too. Please report any problems with 18beta1 (or
discrepancies with other versions) as GitHub issues.

Also, this release requires rust v1.88.0 or greater. `if-let` chains are
now a thing and we're not afraid to use them.

# What's Changed

## Postgres 18beta1 Support

* Support Postgres 18beta1 by @silver-ymz in
pgcentralfoundation#2056
* pg18 support: add header and implement `#define` by @eeeebbbbrrrr in
pgcentralfoundation#2094
* improve pg_magic_func by @usamoi in
pgcentralfoundation#2088


## More Headers

* Added `catalog/heap.h` binding by @ccleve in
pgcentralfoundation#2072
* include `utils/pg_status.h` by @eeeebbbbrrrr in
pgcentralfoundation#2091


## `cargo-pgrx` improvements

* Pass `LLVM_*` variables to `--runas` command by @theory in
pgcentralfoundation#2083
* `does_db_exist()`: fix `psql` argument order by @eeeebbbbrrrr in
pgcentralfoundation#2093
* `cargo pgrx regress` output is no longer fully buffered by
@eeeebbbbrrrr in pgcentralfoundation#2095
* Detect `pgrx_embed` name from lib name by @YohDeadfall in
pgcentralfoundation#2035
* Fixed error message if no artifact found by @YohDeadfall in
pgcentralfoundation#2034
* `cargo-pgrx`: use system certificate store for HTTPS validation by
@charmitro in pgcentralfoundation#2074
* Decoding command output in Windows by @if0ne in
pgcentralfoundation#2084


## Breaking Changes

* fix GUC by @usamoi in
pgcentralfoundation#2064
* refactor GUC by @usamoi in
pgcentralfoundation#2066

## New Stuff

* Added `pg_binary_protocol` attribute to derive send and receive
functions for `PostgresType` by @LucaCappelletti94 in
pgcentralfoundation#2068
* Expose guc hooks by @thesuhas in
pgcentralfoundation#2075
* Allows to create multiple aggregates for the same Rust type by @if0ne
in pgcentralfoundation#2078



## General Code Cleanup

* `cargo clippy --fix` by @eeeebbbbrrrr in
pgcentralfoundation#2092
* Use `if-let` to unpack Options by @stuhood in
pgcentralfoundation#2089
* docs: fix typo in `rust_byte_slice_to_bytea()` docs by @burmecia in
pgcentralfoundation#2071
* Added a missing `#[doc(hidden)]` by @LucaCappelletti94 in
pgcentralfoundation#2079

## Administrative

* Updated Fedora to latest in CI by @YohDeadfall in
pgcentralfoundation#2085
* fix ci on beta rust (1.89) by @usamoi in
pgcentralfoundation#2087

## New Contributors

Much thanks to our new contributors! Your work is sincerely appreciated!

* @charmitro made their first contribution in
pgcentralfoundation#2074
* @thesuhas made their first contribution in
pgcentralfoundation#2075
* @if0ne made their first contribution in
pgcentralfoundation#2084
* @stuhood made their first contribution in
pgcentralfoundation#2089

**Full Changelog**:
pgcentralfoundation/pgrx@v0.14.3...v0.15.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Panic in non-utf-8 code pages in Windows

4 participants

@if0ne @usamoi @YohDeadfall @eeeebbbbrrrr