Skip to content

Commit

Permalink
And NMEA stream to menu
Browse files Browse the repository at this point in the history
  • Loading branch information
@justcallmekoko
justcallmekoko committed Nov 27, 2023
1 parent a704339 commit 007819f
Show file tree
Hide file tree
Showing 4 changed files with 30 additions and 4 deletions.
2 changes: 1 addition & 1 deletion esp32_marauder/CommandLine.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ const char PROGMEM HELP_SETTINGS_CMD[] = "settings [-s <setting> enable/disable>
const char PROGMEM HELP_LS_CMD[] = "ls <directory>";
const char PROGMEM HELP_LED_CMD[] = "led -s <hex color>/-p <rainbow>";
const char PROGMEM HELP_GPS_DATA_CMD[] = "gpsdata";
const char PROGMEM HELP_GPS_CMD[] = "gps [-g] <fix/sat/lon/lat/alt/date/accuracy/nmea>";
const char PROGMEM HELP_GPS_CMD[] = "gps [-g] <fix/sat/lon/lat/alt/date/accuracy/nmea> [-n] <all/gps/glonass/galileo>";
const char PROGMEM HELP_NMEA_CMD[] = "nmea";

// WiFi sniff/scan
Expand Down
1 change: 1 addition & 0 deletions esp32_marauder/GpsInterface.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

#include <MicroNMEA.h>
#include <SoftwareSerial.h>
#include <LinkedList.h>

#include "configs.h"

Expand Down
15 changes: 12 additions & 3 deletions esp32_marauder/MenuFunctions.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -500,7 +500,8 @@ void MenuFunctions::main(uint32_t currentTime)
(wifi_scan_obj.currentScanMode == OTA_UPDATE) ||
(wifi_scan_obj.currentScanMode == ESP_UPDATE) ||
(wifi_scan_obj.currentScanMode == SHOW_INFO) ||
(wifi_scan_obj.currentScanMode == WIFI_SCAN_GPS_DATA)) {
(wifi_scan_obj.currentScanMode == WIFI_SCAN_GPS_DATA) ||
(wifi_scan_obj.currentScanMode == WIFI_SCAN_GPS_NMEA)) {
if (wifi_scan_obj.orient_display) {
this->orientDisplay();
wifi_scan_obj.orient_display = false;
Expand Down Expand Up @@ -558,7 +559,8 @@ void MenuFunctions::main(uint32_t currentTime)
(wifi_scan_obj.currentScanMode != OTA_UPDATE) &&
(wifi_scan_obj.currentScanMode != ESP_UPDATE) &&
(wifi_scan_obj.currentScanMode != SHOW_INFO) &&
(wifi_scan_obj.currentScanMode != WIFI_SCAN_GPS_DATA))
(wifi_scan_obj.currentScanMode != WIFI_SCAN_GPS_DATA) &&
(wifi_scan_obj.currentScanMode != WIFI_SCAN_GPS_NMEA))
{
// Stop the current scan
if ((wifi_scan_obj.currentScanMode == WIFI_SCAN_PROBE) ||
Expand Down Expand Up @@ -618,7 +620,8 @@ void MenuFunctions::main(uint32_t currentTime)
(wifi_scan_obj.currentScanMode != OTA_UPDATE) &&
(wifi_scan_obj.currentScanMode != ESP_UPDATE) &&
(wifi_scan_obj.currentScanMode != SHOW_INFO) &&
(wifi_scan_obj.currentScanMode != WIFI_SCAN_GPS_DATA))
(wifi_scan_obj.currentScanMode != WIFI_SCAN_GPS_DATA) &&
(wifi_scan_obj.currentScanMode != WIFI_SCAN_GPS_NMEA))
{
// Stop the current scan
if ((wifi_scan_obj.currentScanMode == WIFI_SCAN_PROBE) ||
Expand Down Expand Up @@ -1668,6 +1671,12 @@ void MenuFunctions::RunSetup()
wifi_scan_obj.StartScan(WIFI_SCAN_GPS_DATA, TFT_CYAN);
});

this->addNodes(&deviceMenu, "NMEA Stream", TFT_ORANGE, NULL, GPS_MENU, [this]() {
wifi_scan_obj.currentScanMode = WIFI_SCAN_GPS_NMEA;
this->changeMenu(&gpsInfoMenu);
wifi_scan_obj.StartScan(WIFI_SCAN_GPS_NMEA, TFT_ORANGE);
});

// GPS Info Menu
gpsInfoMenu.parentMenu = &deviceMenu;
this->addNodes(&gpsInfoMenu, text09, TFT_LIGHTGREY, NULL, 0, [this]() {
Expand Down
16 changes: 16 additions & 0 deletions esp32_marauder/WiFiScan.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1188,6 +1188,22 @@ void WiFiScan::RunGPSNmea() {
if(nmea_sentence != "" && nmea_sentence != old_nmea_sentence){
old_nmea_sentence=nmea_sentence;
Serial.println(nmea_sentence);
#ifdef HAS_SCREEN
// Get screen position ready
display_obj.tft.setTextWrap(true);
display_obj.tft.setFreeFont(NULL);
display_obj.tft.setCursor(0, SCREEN_HEIGHT / 3);
display_obj.tft.setTextSize(1);
display_obj.tft.setTextColor(TFT_CYAN);

// Clean up screen first
display_obj.tft.fillRect(0, (SCREEN_HEIGHT / 3) - 6, SCREEN_WIDTH, SCREEN_HEIGHT - ((SCREEN_HEIGHT / 3) - 6), TFT_BLACK);

display_obj.tft.setCursor(0, SCREEN_HEIGHT / 3);

display_obj.tft.print(nmea_sentence);
display_obj.tft.setTextWrap(false);
#endif
}
}

Expand Down

15 comments on commit 007819f

@aubreyshomo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI -- your has screen code here is in a fallback that should never be reached.

The else block it's in only triggers if there's no buffer (null pointer) or queue mode is off.

Queue mode is always enabled when nmea scan is initiated. So, this code block, at 1180, which does its best to fix the state it found itself in, shouldn't actually be run in properly executing code.

In properly executing code, there is a buffer and queue is enabled, so it's the loop above that replaces the LL with a clean one then outputs everything then deletes old one that does the packet display.

Then, the two locally generated packets are serial output below the if/else block, in either case.

Not sure if this is just the start, but figured I'd mention that block will only output the most recent non-important (read: not parsed by MicroNMEA) sentence and only under circumstances you'd have to rebuild or h4x0r somehow to cause.

In case you're banging your head into why it's not working.

(no screen, can't test here)

@aubreyshomo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, see why that worked.

You didn't enable_queue() on gps_obj from the menu. That call tells the gspinterface to start queueing up packets for display.

That's why that worked, and possibly why you're not seeing the packets you expected to.

@aubreyshomo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bottom line: That else block is my first try at the relay and it fails.

The GPS emits like 20 (mammal decimal guestimate) sentences every time that light blinks. That else block can only see whichever one came through last.

When that was how I did the relay, I only had enough info for a position update once about every 3 minutes, and I'm guessing that's only due to timer skew making which one was last processed lucky, though whether that's timer skew in the asyncronous parts that move data from serial to nmea to gps_obj and then output from gps_obj when the scan is running, or timer skew that actually reorders output from the chip, I did not explore. Both explanations seem sketchy, but it shouldn't have worked at all.

(That is the reason I added the queue, then I added locally generated messages at the bottom because MicroNMEA eats those messages and won't poop them at my callback without modding it, so I just worked around. This technique adds decimeter flutter to the location, due to precision constraints, but the module is only a 10m accuracy anyways, so I figured I could stop at decimeter and commit.)

My recommendation:

  1. Init the screen above the if/else at 1173
  2. keep "display_obj.tft.print(nmea_sentence);" in the else block
  3. add "display_obj.tft.print(nmea_sentence);" for both gps_obj.generateGXgga() and gps_obj.generateGXrmc() below the if/else, probably stick the String in a local var, call display on it, then just pass local var .c_str to the serial outputters at old lines 1194-5, new lines 1210-1. Those two packets will never be returned by gps_obj.getNmeaNotimp() and have to be locally made.
  4. consider what to do when in queue mode. probably too many to print all.
  5. possible solution for queue mode: move 1187 above 1173 for scope, in if branch at 1173, before else, add "if(nmea_sentence != "") display_obj.tft.print(nmea_sentence);" after deleting buffer. This would output the one packet either way, whatever was last processed, and the two freshly generated ones that actually provide position, and keep the screen from overflowing.
  6. gps_obj.enable_queue() should be called from menu option, same as from cmdline option as it does make serial port output nmea stream too when button pressed. make sure to disable_queue() on any input that stops the scan or the queue will just fill up.

Sorry if I'm being annoying or unhelpful, but I figured I could help you out on adding the screen stuff from a this-is-what-my-code-does perspective.

I'd have added that, but don't have a screen.

@justcallmekoko
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aubreyshomo
Let me know if this is what you're talking about. I added the fixes you were referring to in 1, 2, 4, and 6 but I'm not sure if I understand queue mode enough to implement 5
7a15b5b

I'm am getting better print messages to the screen and seeing the data in the Serial monitor. The only thing I am not seeing on screen in the GPTXT message regarding the ANTENNA status

@aubreyshomo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Glancing through everything. but very tired. was up all night migrating data for CMS system. must sleep now. reply later.

@aubreyshomo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@justcallmekoko: FYI, here's a flowchart for the queueing mechanism.

ESP32 NMEA Queueing

The bottom line is MicroNMEA provides a callback to give me the sentence iff (if and only if) it does not understand it. Then it goes in the queue.

If it does understand it, it just updates position in the nmea object, which is then updated into gps_obj on a regular basis.

(This is why we fake those two messages it understands ourselves, as I will document in the next comment / picture.)

@aubreyshomo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only thing I am not seeing on screen in the GPTXT message regarding the ANTENNA status

That goes into the queue and into notimp_nmea_sentence and into nmea_sentence.

So, if the queue is on, you should have that in there.

If the queue is off, then it could possibly be in notimp_nmea_sentence or nmea_sentence.

The thing with those is each stores only one sentence (the most recent) and the GPS sends many sentences at once. So it's luck of the draw if the GPSTXT is in the most recent slot in the object.

The difference is notimp_nmea_sentence specifically does not store GxGGA or GxRMC sentences, since it is populated from the not understood callback after every message is processed.

nmea_sentence is populated when GpsInterface::setGPSInfo() is called, and will have the most recent NMEA sentence processed by the nmea object as of the time that is run, which could be the same sentence as in notimp_nmea_sentence, or the same type, but might also be the orignal gxgga or gxrmc messages, as this one is not filtered for those, but, again, it's really random which one you'll have. the most recent as of the run. a tad racey which, but will have one of them.

so, without the queue, you only get the most recent sentence you know you won't be regenerating, or the most recent sentence without that filter. you don't get every sentence.

perhaps gptxt just got unlucky? It should have the same precedence and processing as anything not GxRMC or GxGGA.

And that's a limitation of the MicroNMEA lib.

@aubreyshomo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is a flowchart for the output process:

ESP32 NMEA Output

As you can see, we output the whole queue if queue mode is on.

If queue mode is off, we output the most recently processed NMEA sentence that is not a GxGGA or GxRMC (gps_obj.notimp_nmea_sentence). We use this one, not the one that is unfiltered, because of the final step.

Because we can't get raw GxGGA or GxRMC sentences from MicroNMEA, we make our own from the position stored in the gps_obj object, and output those instead. (We do this whether or not we're in queue mode. These two sentences will never be in the queue due to MicroNMEA limitations.)

So, ugly hack? A bit.

Does it work? Yes, it does.

@aubreyshomo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@justcallmekoko : I also threw down for some en-screened marauder hardware, so I can check this out on the real deal.

@aubreyshomo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@justcallmekoko: Lastly, looked over everything. #4 and #5 involve a decision about what to do with the too much data in the queue.

I tried to document the queueing mechanism and what it does above, as well as how I handle the output.

So, with queue off, you'll get the GGA and RMC messages on the screen, as well as the most recent other message type. This works. (your edits for #1,2,3,6 look good to me.)

I did, however, have a thought for #6. We could put the queue_enable call in the wifi_scan_obj's startscan method, if it's a NMEA scan, and remove it everywhere else. Then it always gets called to enable for nmea scans, and there's no need to call it anywhere you are about to call startscan anyways.

Just a late breaking thought, but I figure it's most elegant there and once, rather than many places.

As for #4/#5, it really comes down to what makes sense on a screen smaller than the data.

Here's a sample of what I get on your GPS/SD card mod's chip. Other chips may have other sets of sentences, but this is a decent example.

With queue on, I count 11 additional NMEA sentences that come through every 1 Hz cycle:

3 GPGSV, reflecting 3 lines of GPS satellites and RSSI figures for each. I think it's four sats per line, so it can be 4 lines. My sample I'm looking at right now isn't, but 12 sentences with the extra being another GPGSV is totally plausible.

2 BDGSV, reflecting 2 lines of Beidou satellites and RSSI figures. This could similarly grow and increase the total.

One each of GPGSA and BDGSA, describing satellites used in the position solution from each constellation, GPS and Beidou, respectively.

One GNVTG message, which provides course and speed information. (Largely redundant to GxRMC, though more detailed.)

One GNZDA message, which provides time and date. (Largely redundant to GxRMC, which provides both, but not local time zone, potentially. Time, but not date, is also in GxGGA.)

One GPTXT message, with the "ANTENNA OK" message.

One GNGLL message, which contains Geographic Position, redundant to GxGGA.

Note with respect to non-parsed position data: This (GxGLL) is directly from the GPS device, whereas our GxGGA is processed and reconstituted. This can lead to decimeter flutter between the GLL message, or any similar non-parsed message that contains position information, and the marauder produced GGA message. This is because we have 5 digits of precision in decimal minutes, after the decimal, plus two or three in front, depending on lat/lng, but MicroNMEA only has millionths of a degree precision. One millionth of a degree is 0.00006 decimal minutes, meaning the last significant figure has only half an order of magnitude precision after error bars.

This means the GPS, using all 5 decimal places, can be 11.1 cm, or about a decimeter, more accurate than we can.

So, if your application processes both, there will be decimeter flutter depending on which message was processed last in any group. It is for this reason we always send our homemade GGA and RMC packets last.

Again, this is just what I get from the IC you picked for your mod. Other GPSs will have slightly different messages.

This leads to two real options:

  1. Decide which of that stuff is worth throwing on the screen, and then process to filter for that, from the queue
  2. Ignore the queue and just output the notimp_nmea_sentence like we do when the queue is off. This will be one random non-parsed packet along with the two we reconstitute on the screen, then, exactly the same as in non-queued mode.

To implement option #2:

a. move "String nmea_sentence=gps_obj.getNmeaNotimp();" above "if(buffer && gps_obj.queue_enabled()){" to broaden scope.
b. add "display_obj.tft.print(nmea_sentence);" after "delete buffer;" in the queue mode block, also

That'll at least have it work the same in queue mode and not, until a good decision can be made and implemented for option #1, if desired.

@aubreyshomo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anyways, sorry it took me so long to get back to you, but hopefully this answers all your questions, and fills in the blanks about the queueing and reconstituting mechanism and why I did it that way (hack around library limitations).

Shout if you've got any questions / thoughts / whatever.

@aubreyshomo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

next day thoughts:

"This is because we have 5 digits of precision in decimal minutes, after the decimal, plus two or three in front, depending on lat/lng, but MicroNMEA only has millionths of a degree precision.

I'm sort of 1D10T'd this statement. Always two digits in front, it's decimal minutes, not degrees, where it can be two N/S, or 3 E/W. Always fixed point precision for us 2.5 because at most 60 minutes. GPS devices can vary, but above 5 decimal places is rare to be precise, becoming more common, but not quite there yet, and 5 decimal places just fits as much data as we can get out of MicroNMEA, precision wise, just only multiples of 610^-5 mod 110^-4 in that last digit. Your mod provides 5 decimal places of precision, for instance.

Probably clear what I meant, but I was being an idiot there.

Ignore the queue and just output the notimp_nmea_sentence like we do when the queue is off. This will be one random non-parsed packet along with the two we reconstitute on the screen, then, exactly the same as in non-queued mode.

This is basically my original suggestion #5. At least until we have a better option for what to display. Then it at least works, though resolution on extra data (non GGA/RMC) quite low and selection displayed random.

Here is a flowchart for the output process:

Improved slightly for clarity.

ESP32 NMEA Output v2

Made it clear we're using the marauder's updated position, from gps_obj, for the homemade packets.

The only thing I am not seeing on screen in the GPTXT message regarding the ANTENNA status

I was thinking further about this, and I think the GPTXT is the single most important addition to the two homemade messages on the screen.

You could always do a bounded string compare on the first for "$" and 4th through 7th for "TXT," one based, in the callback (gps_obj.enqueue()), and put that in an additional text buffer, for output to the screen.

I just found GPTXT way handy on mine earlier today. I had a new antenna that just wouldn't work.

$GPTXT was "ANTENNA OPEN".

Now I have a return I need to make. Thing was screwed into the SMA tight, and the SMA was plugged into the board right. Confirmed with a known-good.

So, in practice, I think you hit on that being very important and probably the first thing to handle for option #1 or to add to the random sentence from option #2.

Thoughts / tweaks / improvements / clarifications from the next day...

@aubreyshomo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@justcallmekoko just got a marauder v6 and a mini. Neither had gps built in. I'll probably add it to the v6. I'm not sure I'm up to cramming one in the mini, but I haven't opened either yet. Just got them.

Either way, I have a screen to target, so I'll just hard code a mock and see what I can make the screen do and send this branch a pull request.

Any advice on sticking a gps mod in them?

@aubreyshomo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@justcallmekoko : Saw you just merged NMEA into master.

I've got another pull request for you, here, shortly. Adds three other satalite systems, some UI niceness, and parsing of GPTXT messages with their own queue for display and for inclusion in gpsdata or gps -g text queries.

So, I've got a decent starting place for an answer for #5 I'm about to push your way.

Found this: https://github.com/justcallmekoko/ESP32Marauder/wiki/gps-modification

Bought mods. Have not tested on a real screen yet, but fully tested on flipper devboard.

@aubreyshomo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check it out:

`--------------------------------

     ESP32 Marauder

        v0.13.6

   By: justcallmekoko

#gps -g text
ANTENNA OK

`

Please sign in to comment.