Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
161 commits
Select commit Hold shift + click to select a range
2c9a19a
First cut at calling the new nceplibs bufr query interface.
rmclaren Jul 14, 2021
911ab75
Updated to handle multi-dimensional data. Fixed DatetimeVariable. Got…
rmclaren Jul 16, 2021
7292097
Added support for CCITT IA5 string fields. Fixed many other fields th…
rmclaren Jul 26, 2021
4197505
Added support for forfield. Cleaned up some uneccesarrily complicated…
rmclaren Jul 26, 2021
63ccb24
Removed split map in favor of a list of splits to make code more read…
rmclaren Jul 27, 2021
0267205
Fixed bug with transforms. Updated some YAML files.
rmclaren Jul 28, 2021
2fe07fc
Updated the README file.
rmclaren Jul 28, 2021
e6a57ed
Fixed bug were CCITT IA5 fields would fail because of a silly mistake…
rmclaren Aug 5, 2021
0bcdf01
Merge branch 'develop' into feature/query
emilyhcliu Aug 14, 2021
569c0d4
Renamed for field to group_by field.
rmclaren Aug 18, 2021
a28c35c
Merge branch 'feature/query' of https://github.com/JCSDA-internal/iod…
rmclaren Aug 18, 2021
0c2dd81
Removed duplicate directory for transforms.
rmclaren Aug 20, 2021
9bf4a30
Merge branch 'feature/query' of https://github.com/JCSDA-internal/iod…
rmclaren Sep 23, 2021
be4c663
incremental checkin, getting higher dimensional data working
rmclaren Sep 30, 2021
df685f0
Added DataObject cpp
rmclaren Oct 1, 2021
5c40747
Finished major code revisions and got things to compile for automatic…
rmclaren Oct 15, 2021
2ef38f9
Some bug fixes to get simple radiance data working.
rmclaren Oct 18, 2021
a30979d
Enhanced the reading of th dimensions field.
rmclaren Oct 18, 2021
3aad615
Got filtering and splitting to work again. Might need to be a little …
rmclaren Oct 18, 2021
80dbe6e
Fixed a dimension naming issue with the way group_by location dimensi…
rmclaren Oct 19, 2021
129fb53
Added arguments to bufr2ioda so you can limit the amount of messages …
rmclaren Oct 20, 2021
4c37c3b
Changed dimensions attribute to list or single string.
rmclaren Oct 21, 2021
ec23950
Fixed transforms.
rmclaren Nov 4, 2021
6de09ed
Fixed transfoms so they will not affect missing values.
rmclaren Nov 4, 2021
9f7b8fa
Fixed bug that happened when we tried to export an empty category.
rmclaren Nov 5, 2021
b165236
Fixed strange issue that happened while applying filters. Removed som…
rmclaren Nov 9, 2021
0051a0e
Updated a bunch of unit tests so that they would work with the latest…
rmclaren Nov 16, 2021
c655f4a
updated test output files to work with latest version of ioda
rmclaren Nov 18, 2021
8644963
Added unit test and now time the runtime.
rmclaren Dec 17, 2021
25982ab
Fixed a couple more unit tests.
rmclaren Dec 21, 2021
2b81f86
Added .dump files to gitlfs
rmclaren Dec 22, 2021
0dabeb8
removed uneeded file
rmclaren Dec 22, 2021
9eb5f62
BUFR2IODA conversion for AMSU-A and MHS Radiance data (#713)
emilyhcliu Dec 22, 2021
1679444
Update README.md
rmclaren Dec 28, 2021
ef43b58
Test and enhance two yaml files for adpsfc snow and sevcrs data (#736)
YoulongXia-NOAA Dec 29, 2021
7dfe9ef
Reduce the size of test files for amsua, mhs and iasi. Update the co…
emilyhcliu Jan 27, 2022
92aedb7
Merged in latest changes from develop.
rmclaren Jan 31, 2022
a48e36a
Removed uneeded files I missed during merge.
rmclaren Jan 31, 2022
aa1f4ff
Initial checkin of query classes with a lot of placeholder code
rmclaren Feb 2, 2022
a723efc
Got data collection mostly working.
rmclaren Feb 8, 2022
f655842
Starting to work end to end. Need to fix exportdims and revisit code …
rmclaren Feb 9, 2022
a815020
Merge branch 'feature/query' of https://github.com/JCSDA-internal/iod…
rmclaren Feb 10, 2022
a9983fc
Fixed issue where targets and masks were being copied unecessarily.
rmclaren Feb 10, 2022
af1304a
Feature/adpupa prepbufr (#794)
PraveenKumar-NOAA Feb 11, 2022
48f5189
Got a bunch of unit tests working.
rmclaren Feb 11, 2022
eebbf26
add aircft profiles to feature/query
nicholasesposito Feb 16, 2022
81a63a1
Fixed non-working functional unit tests.
rmclaren Mar 4, 2022
3432e9c
Merge branch 'feature/query' of https://github.com/JCSDA-internal/iod…
rmclaren Mar 4, 2022
6402f03
Merge branch 'feature/query' into feature/query_cxx
rmclaren Mar 4, 2022
2159242
Added unit test for simple groupby example.
rmclaren Mar 8, 2022
ee81939
Merge branch 'feature/query' of https://github.com/JCSDA-internal/iod…
rmclaren Mar 8, 2022
69a2f72
Merge branch 'feature/query' of https://github.com/JCSDA-internal/iod…
rmclaren Mar 9, 2022
7378b39
Merge branch 'feature/query' into feature/query_cxx
rmclaren Mar 9, 2022
598ebf6
Made simple groupby test work. Fixed related bugs.
rmclaren Mar 10, 2022
2406042
Added unit test for bufr files with missing fields.
rmclaren Mar 11, 2022
d8240c0
Added unit test for sfcshp data
rmclaren Mar 13, 2022
81931e3
accidentaly messed up the bufr-mhs test. this fixes it
rmclaren Mar 14, 2022
430850c
Integrated latest bug fixes from the feature/query branch.
rmclaren Mar 14, 2022
f083af6
Merge branch 'feature/query' of https://github.com/JCSDA-internal/iod…
rmclaren Mar 14, 2022
85c1a3e
Merge branch 'feature/query' into feature/query_cxx
rmclaren Mar 14, 2022
d16bf12
Fixed bug were we weren't checking for empty sequence counts.
rmclaren Mar 14, 2022
6e158c7
added adpupa tests
rmclaren Mar 14, 2022
68bc140
Merge branch 'feature/query' of https://github.com/JCSDA-internal/iod…
rmclaren Mar 14, 2022
6c9f1d6
Merge branch 'feature/query' into feature/query_cxx
rmclaren Mar 14, 2022
eeb0896
Fixed bugs that prevented adpupa from working.
rmclaren Mar 15, 2022
552aceb
Getting much closer to having adpupa groupby example working.
rmclaren Mar 22, 2022
d74796c
fixed final bug to get adpupa groupby case to work the same way.
rmclaren Mar 22, 2022
26ab396
Removed some uneeded code.
rmclaren Mar 22, 2022
56a452c
Merged latest from develop, and made it work with the latest from NCE…
rmclaren Mar 23, 2022
b86c4da
Reduce the size of the ADPUPA test input furrther
emilyhcliu Mar 23, 2022
9da4fb3
Merge branch 'feature/query_cxx' of https://github.com/JCSDA-internal…
rmclaren Mar 30, 2022
da9c483
Removed undesired include from legacy bufr code.
rmclaren Mar 31, 2022
5153474
Removed undesired include from legacy bufr code.
rmclaren Mar 31, 2022
8300c35
Removed undesired include from legacy bufr code.
rmclaren Mar 31, 2022
a052afb
Removed undesired include from legacy bufr code.
rmclaren Mar 31, 2022
418b1f4
Removed undesired include from legacy bufr code.
rmclaren Mar 31, 2022
e83ff00
Removed undesired include from legacy bufr code. Fixed some copyright…
rmclaren Mar 31, 2022
6a892fd
Changed the way data provider works (no longer a singleton). Minor cl…
rmclaren Apr 4, 2022
c683b4b
Added print_queries tool to the ioda-converter project.
rmclaren Apr 5, 2022
fb5f07f
Improved situation arrising from missing string fields.
rmclaren Apr 6, 2022
067390d
Improved detection of string fields.
rmclaren Apr 6, 2022
efa2763
Moved table logic out of the print_quiries function. Fixed bug were i…
rmclaren Apr 8, 2022
0a79cb3
Merge branch 'develop' into feature/query_cxx
rmclaren Apr 11, 2022
ccbbeb6
Replace nlocs with Location, reduce the size of ADPUPA.prepbufr and ……
emilyhcliu Apr 11, 2022
f09a98c
Merge branch 'feature/query_cxx' of https://github.com/JCSDA-internal…
rmclaren Apr 11, 2022
d403b75
Reverted ADPUPA bufr file to smaller one.
rmclaren Apr 11, 2022
dff7251
Fixed compiler warnings
rmclaren Apr 11, 2022
eca2524
Fixed code style (norm) issues.
rmclaren Apr 11, 2022
d723ac0
Added doc strings to query code.
rmclaren Apr 12, 2022
20eb76b
Feature/query cxx praveen (#893)
PraveenKumar-NOAA Apr 13, 2022
6fe7e43
Added white space
rmclaren Apr 13, 2022
664c2bf
Change dimension name to following unified IODA naming convention (#…
emilyhcliu Apr 13, 2022
394ae09
Merge branch 'feature/query_cxx' of https://github.com/JCSDA-internal…
rmclaren Apr 13, 2022
b5d4cdc
Fixed bug when there were more than one subset in the file. Added abi…
rmclaren Apr 18, 2022
b55c75d
Updated print queries so it no longer uses colors but prints the dime…
rmclaren Apr 25, 2022
b98c09d
Chenged some maps to unordered_maps as they are better performance wise.
rmclaren Apr 26, 2022
3122f9a
Feature/query cxx as (#914)
AshleyStanfield-NOAA May 6, 2022
da75624
Modify unit attribute in BUFR2IODA converter and bug fix for satellit…
emilyhcliu May 9, 2022
24d6a3e
Fixed issue where non-existing query with index would cause a crash. …
rmclaren May 17, 2022
68e94ec
Fixed some norm test failures.
rmclaren May 17, 2022
6a5043e
Fixed all the unit tests
rmclaren May 18, 2022
724b104
Made change so that there is always one group_by field whcih applies …
rmclaren May 23, 2022
9657fb2
Removed uneeded test
rmclaren May 23, 2022
107376f
Merge branch 'develop' into feature/query_cxx
emilyhcliu May 24, 2022
f76dbc8
Improved some of the code to do with finding the data object to group…
rmclaren May 24, 2022
bcda337
Merge branch 'feature/query_cxx' of https://github.com/JCSDA-internal…
rmclaren May 24, 2022
c80d02c
Fixed bug in print queries were fixed repeats were being missed.
rmclaren May 25, 2022
29a65b3
update two yaml files via query_cxx (#906)
YoulongXia-NOAA May 26, 2022
9325c5b
Feature/query cxx mhs (#960)
emilyhcliu Jun 2, 2022
a0dd8e1
Feature/query cxx amsua (#958)
emilyhcliu Jun 2, 2022
4bd2e8c
Feature/query cxx iasi (#966)
emilyhcliu Jun 7, 2022
ac14e75
Feature/query cxx nick e adpsfc (#904)
nicholasesposito Jun 7, 2022
f611e3a
Feature/query cxx atovs (#963)
emilyhcliu Jun 14, 2022
afbcb8f
Merge branch 'develop' into feature/query_cxx
srherbener Jun 17, 2022
297b19a
Updated DatetimeVariable.cpp code for optional value of Minute (#949)
PraveenKumar-NOAA Jul 12, 2022
2be7961
Merged latest from develop branch
rmclaren Aug 8, 2022
d7d3f91
Merge branch 'develop' into feature/query_cxx
srherbener Aug 9, 2022
aafc721
Now uses OOPS logging.
rmclaren Aug 12, 2022
df7d5c8
Merge branch 'feature/query_cxx' of https://github.com/JCSDA-internal…
rmclaren Aug 12, 2022
a53f0e1
Fixed some comments
rmclaren Aug 22, 2022
03bbffd
cast time to seconds instead of dividing by 1000
rmclaren Aug 22, 2022
dccf5a4
fixed norm test failure
rmclaren Aug 22, 2022
a76894e
made exit into exception
rmclaren Aug 22, 2022
dfc55d7
using back inserter to append to sequence path
rmclaren Aug 22, 2022
fbadda3
fixed error message.
rmclaren Aug 22, 2022
703b087
fixed error message.
rmclaren Aug 22, 2022
482f684
added prepbufr to git lfs
rmclaren Aug 22, 2022
aff8124
removed line continuation characters.
rmclaren Aug 22, 2022
bdd5c9b
replaced 10e10 with missing value
rmclaren Aug 22, 2022
f3fe27e
accidentally deleted line of code. added it back
rmclaren Aug 22, 2022
adf9c7e
changed print time function back tot he original.
rmclaren Aug 22, 2022
a2f7e65
using gsl::narrow instead of simpoe static cast
rmclaren Aug 22, 2022
39bfe36
Enhanced vector product function
rmclaren Aug 22, 2022
ec02de9
Pulled out missing value into a Constants.h file to avoid duplication.
rmclaren Aug 22, 2022
d4905fe
fixed some comments
rmclaren Aug 22, 2022
509af0f
fixed file permissions
rmclaren Aug 22, 2022
03dadc4
Added try catch statements to main functions. Fixed warning.
rmclaren Aug 22, 2022
06fb678
pre reserve some memory
rmclaren Aug 23, 2022
79ad913
removed unused test from CMakeLists.txt file.
rmclaren Aug 23, 2022
8dff03a
Added include of unordered map as some compilers were not finding it.
rmclaren Aug 29, 2022
d823c86
merged latest from develop
rmclaren Aug 31, 2022
a5a1b1b
Reverted unintended changes to files in ncar-bufr2nc-fortran and lib-…
rmclaren Aug 31, 2022
4c43af3
Fixed bug that invovled parsing the bufr table data incorrectly. Basi…
rmclaren Sep 1, 2022
f4477bf
Fixed bug in print queries as it was not trimming subset trims proper…
rmclaren Sep 1, 2022
71f2046
Fixed some warnings and errors that happen with the CLANG compiler.
rmclaren Sep 1, 2022
5281c18
Merge branch 'develop' into feature/query_cxx
srherbener Sep 1, 2022
ba1b34b
Enable the C++14 standard for C++ compiles. (#1036)
srherbener Sep 1, 2022
2cf589b
Fixed some code review issues that Cory Martin broght up.
rmclaren Sep 2, 2022
e49879a
Merge branch 'feature/query_cxx' of https://github.com/JCSDA-internal…
rmclaren Sep 2, 2022
71d2c26
updated docs for DataObject::print function
rmclaren Sep 2, 2022
c78b93e
commented in line that was accidentally commented out
rmclaren Sep 2, 2022
6c29f1f
Revert "commented in line that was accidentally commented out"
rmclaren Sep 2, 2022
f92e4e5
Added back missing tests from the develop branch.
rmclaren Sep 2, 2022
cdecff3
Moved test_iodaconv_generic_gnssro_bufr back to where it was before (…
rmclaren Sep 2, 2022
c5034db
Forcing named dimensions to be generic paths (not based on subset). B…
rmclaren Sep 6, 2022
25b31d9
Merge branch 'develop' into feature/query_cxx
CoryMartin-NOAA Sep 8, 2022
09c01c7
Fixed issue caused by code review comment fix that caused bufr2ioda.x…
rmclaren Sep 9, 2022
4e14eb9
Fixed exception handler in print_queries.x.
rmclaren Sep 12, 2022
30b1a0f
Merge branch 'feature/query_cxx' into feature/query_dim_naming_bug
rmclaren Sep 12, 2022
d994a02
merged in latest from develop
rmclaren Sep 21, 2022
1d17108
Merge branch 'develop' into feature/query_dim_naming_bug
PatNichols Sep 26, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions src/bufr/IodaEncoder/IodaEncoder.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,16 @@ namespace Ingester
+ path);
}

if (path.substr(0, 1) != "*")
{
std::ostringstream errStr;
errStr << "ioda::dimensions: ";
errStr << "Path " << path << " must start with *. ";
errStr << "Subset specific named dimensions are not supported.";

throw eckit::BadParameter(errStr.str());
}

dimPaths.insert(path);
}

Expand Down
3 changes: 1 addition & 2 deletions test/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -939,7 +939,7 @@ ecbuild_add_test( TARGET test_${PROJECT_NAME}_mopitt_co
-i testinput/mopitt_co.he5
-o testrun/mopitt_co.nc
-r 2021092903 2021092921"
mopitt_co.nc ${IODA_CONV_COMP_TOL_ZERO})
mopitt_co.nc ${IODA_CONV_COMP_TOL_ZERO})

ecbuild_add_test( TARGET test_${PROJECT_NAME}_modis_aod
TYPE SCRIPT
Expand Down Expand Up @@ -1201,7 +1201,6 @@ if(iodaconv_bufr_ENABLED)
ecbuild_add_test( TARGET test_iodaconv_bufr_adpupa
TYPE SCRIPT
COMMAND bash
COMMAND bash
ARGS ${CMAKE_BINARY_DIR}/bin/iodaconv_comp.sh
netcdf
"${CMAKE_BINARY_DIR}/bin/bufr2ioda.x testinput/adpupa_prepbufr.yaml"
Expand Down
6 changes: 3 additions & 3 deletions tools/bufr/print_queries.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ getDimPaths(const std::vector<Ingester::bufr::QueryData>& queryData)
for (auto& query : queryData)
{
std::stringstream pathStream;
pathStream << query.pathComponents[0];
pathStream << "*";
for (size_t idx=1; idx <= query.dimIdxs.back(); idx++)
{
pathStream << "/" << query.pathComponents[idx];
Expand Down Expand Up @@ -124,7 +124,7 @@ void printDimPaths(std::vector<std::pair<int, std::string>> dimPaths)
{
for (auto& dimPath : dimPaths)
{
std::cout << " " << dimPath.first << "d " << dimPath.second << std::endl;
std::cout << " " << dimPath.first << "d " << dimPath.second << std::endl;
}
}

Expand All @@ -133,7 +133,7 @@ void printQueryList(const std::vector<Ingester::bufr::QueryData>& queries)
for (auto query : queries)
{
std::ostringstream ostr;
ostr << dimStyledStr(query.dimIdxs.size()) << " ";
ostr << dimStyledStr(query.dimIdxs.size()) << " ";
ostr << query.pathComponents[0];
for (size_t pathIdx = 1; pathIdx < query.pathComponents.size(); pathIdx++)
{
Expand Down