Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure PIPCamera maintains render state in NoDisplay #2518

Merged
merged 1 commit into from
Apr 3, 2020

Conversation

saihv
Copy link
Contributor

@saihv saihv commented Apr 3, 2020

Fixes a bug with "ViewMode": "NoDisplay" where the FPV camera would disregard both intrinsic (e.g. auto exposure settings in json) and extrinsic (e.g. world postprocessing volume) render settings. PIPCamera blueprint now has the bAlwaysPersistRenderingState flag turned on so the render state persists even when the main viewport render is disabled.

Sample settings.json:

{
  "SettingsVersion": 1.2,
  "SimMode": "ComputerVision",
  "ViewMode": "NoDisplay",
  "CameraDefaults": {
    "CaptureSettings": [
      {
        "ImageType": 0,
        "Width": 640,
        "Height": 480,
        "AutoExposureSpeed": 100,
        "AutoExposureBias": 10,
        "AutoExposureMaxBrightness": 0.64,
        "AutoExposureMinBrightness": 0.03
      }
    ]
  }
}

Sample script capture.py:

import airsim
import numpy as np
import cv2

client = airsim.VehicleClient()

while True:
    rawImage = client.simGetImage("0", airsim.ImageType.Scene)
    if (rawImage == None):
        print("Camera is not returning image, please check airsim for error messages")
        sys.exit(0)
    else:
        png = cv2.imdecode(airsim.string_to_uint8_array(rawImage), cv2.IMREAD_UNCHANGED)
        cv2.imshow("RGB", png)
    
    key = cv2.waitKey(1) & 0xFF;
    if (key == 27 or key == ord('q') or key == ord('x')):
        break;

Steps to test:

  1. Run the game in Blocks, ensure main viewport is blank.
  2. Run capture.py and ensure image is washed out (reflecting "AutoExposureBias": 10 in settings).
  3. Set AutoExposureBias to 0, run capture.py and ensure image looks normal.
  4. Select PostProcessVolume in Blocks and increase Details->Lens->Exposure->ExposureCompensation to 8.0 (maximum).
  5. Run capture.py again and ensure image is washed out.

@saihv saihv merged commit 9edc107 into microsoft:master Apr 3, 2020
@saihv saihv deleted the PR/PP_nodisplay branch April 3, 2020 18:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant