Airsim crash while training

[CoolCoder54323]

Bug report

• AirSim Version/#commit: 1.3.1
• UE/Unity version: 4.25.4
• OS Version: windows 10

What's the issue you encountered?

The simulation crashes after any amount of training time, could be 2k steps or 100k. It gives this error:

Settings

{
"SeeDocsAt": "https://github.com/Microsoft/AirSim/blob/master/docs/settings_json.md",
"SettingsVersion": 1.2,
"SimMode": "Multirotor",
"ClockSpeed": 1.0,
"Vehicles": {
"SimpleFlight": {
"VehicleType": "SimpleFlight",
"DefaultVehicleState": "Armed",
"EnableCollisionPassthrogh": false,
"EnableCollisions": true,
"AllowAPIAlways": true,
"RC": {
"RemoteControlID": 0,
"AllowAPIWhenDisconnected": false
}
}
}
}

How can the issue be reproduced?

It happens every time I train I have yet to get a full session in.

Include full error message in text form

Unhandled exception at 0x00007FF981AA96AD (UE4Editor-AirSim-Win64-DebugGame.dll) in UE4Editor-Win64-DebugGame.exe: 0xC0000005: Access violation reading location 0x0000000000009074.

[jonyMarino]

Hi @CoolCoder54323! Can you provide more details on how you are training the drone so we can reproduce the issue?

[CoolCoder54323]

I am using the rl files that came with airsim I modeified the dron_env in airgym and made a custom file to run it called dronetrain
I also moved airgym into pythonclient/multirotor and that is all the changes I made.

Drone_Env:
drone_env.txt

DroneTrain:
DroneTrain.txt
(I didnt know how you wanted me to upload my code so I jsut used text files since github doesnt support .py files)

[CoolCoder54323]

One more thing to add. It usually happens when I click into unreal engine after being tabbed out and interact with it, but not always it happened one time when I already had UE in focus and I was away from my computer. Hope this helps!

[CoolCoder54323]

@jonyMarino Were you by any chance able to reproduce the error.

[saihv]

@CoolCoder54323 Could you please run the game in DebugGameEditor mode (in Visual Studio), and show a screenshot of the call stack that appears in the bottom right when the crash happens?

[CoolCoder54323]

Ok, no problem my take a few hours considering when it crashed in the past. Kind of annoying though because I remember looking at that and for some reason, I didn't screenshot it.

[CoolCoder54323]

@CoolCoder54323 Could you please run the game in DebugGameEditor mode (in Visual Studio), and show a screenshot of the call stack that appears in the bottom right when the crash happens?

Here you go @saihv. It looks like it came from an unreal file RenderRequest. I wonder if this is a problem with unreal and not Airsim.

[saihv]

This is quite odd, I don't think I've come across this issue before. It seems like something is going wrong with the recording/image capture functionality. Are you using the Blocks environment?

And I see you mentioned you were using 1.3.1, but not the binary, so are you working with an older commit? If yes, perhaps you could retry with the latest master?

[CoolCoder54323]

I am using my own custom environment not blocks, and I don't know what you mean by binary (sorry I am new to this), but I last updated airsim was nov 25. I will try to reinstall and see what happens.

[ahmed-elsaharti]

I don't know what you mean by binary

@CoolCoder54323 binaries are the pre-compiled Unreal Engine environments found in the releases section. Those are ready-to-go executable environments that were built with the latest (at the time of release) Airsim commit

[CoolCoder54323]

So do you think It is because of my environment? I will try it on blocks and see what happens.

[Gastastrophe]

This apparently has been an unresolved issue for quite some time. I believe it is the same issue as #701, #2926, #2196, #2175, and most importantly #329. All of these issues were closed without proper resolution. In #329, @saihv stated that setting bHDR to false in APIPCamera::updateCaptureComponentSettings seemed to fix the issue. It apparently happens because HDR images are very large and flushing them is taxing on Unreal. This means that taking lots of images at a rapid pace (like in your reinforcement learning task) or through multiple drones at the same time (as in #329) causes the breakpoint. PR #2926 was meant to address this, but was never completed without errors.

I also use a variant of the dqn_drone.py script, and I would love to hear how we can do the fix from #329 in Python.

EDIT: I did the change suggested in #329, and I am still having the same problem.

[Gastastrophe]

I also want to report that this issue only happens for Scene images, so taking Depth Perspective images like in dqn_drone.py seems to work fine. I'm going to try to make an easily reproducible version of the bug and post a new issue about it.