Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issue with restart TWS #245

Open
akatsev opened this issue Dec 23, 2023 · 22 comments
Open

Issue with restart TWS #245

akatsev opened this issue Dec 23, 2023 · 22 comments

Comments

@akatsev
Copy link

akatsev commented Dec 23, 2023

Hi,

Who can explain what had happened here?

Thanks a lot in advance,
Aryeh Katsev

Issue
IBC-3.18.0_TWS-1026_FRIDAY.txt.zip
@rlktradewright
Copy link
Member

First of all, you need to find out why TWS is having trouble writing to its logfile. All those messages saying Couldn't write to log file - java.io.IOException: write beyond end of stream are generated by TWS writing to stderr, which then get put into the IBC logfile. I don't know what the cause of this is: it's nothing to do with IBC.

TWS writes log info initially to a file called C:\Jts\1026\launcher.log. Once logon has succeeded, it switches to an encrypted logfile in C:\Jts\1026\cjmgpffcjehgdphobpeddjggomchmllnbpkogpfp. Perhaps looking in launcher.log might give you some clues. If you can't find the source of this issue, then you'll need to follow it up with IBKR.

Why are you running IBC/TWS under Administrator? There is absolutely no good reason to do this, and it's generally bad practice to do anything under Administrator, especially on a server. My advice would be to set up a standard unprivileged user for running IBC/TWS.

The next thing to note is that the auto-restart did actually succeed: you can see this in line 355000:

2023-12-23 00:01:52:717 IBC: Login has completed

It then ran ok for some time: I can't tell when the trouble started because it looks like you've deleted a whole lot of entries from the file, seeing as at 2023-12-23 05:24:52:599 it says Attempt 47: Authenticating...;, but there's no mention of the previous 46 attempts!

So I really can't tell what's happened here: later log entries saying Attempt 47: server error, will retry in seconds... seem to indicate that there is a fault at the server end, but no indication of what that might be.

Failures at the IBKR server are not something IBC can be expected to do anyting sensible about. It would be possible for IBC to recognise a dialog with server error in the title and then cause IBC to exit and restart, but I'm reluctant to do this without more information about the problem.

@akatsev
Copy link
Author

akatsev commented Dec 24, 2023 via email

@akatsev
Copy link
Author

akatsev commented Dec 27, 2023 via email

@rlktradewright
Copy link
Member

No, what IBKR say doesn't help at all.

But is this still occuring? Your issue #246 doesn't seem to have the same problem.

@akatsev
Copy link
Author

akatsev commented Dec 28, 2023 via email

@rlktradewright
Copy link
Member

I'm sorry I dojn't really understand what you're saying. What do you mean by 'I set the ETC time on the computer'? And 'TWS shows TWS tiime' - again i'm not sure what you mean.

Probably the simples thing to do is to edit the file C:\Jts\jts.ini, and replace the TimeZone setting in the [Logon] section with US/Eastern.

@akatsev
Copy link
Author

akatsev commented Dec 28, 2023 via email

@akatsev
Copy link
Author

akatsev commented Dec 29, 2023 via email

@rlktradewright
Copy link
Member

I've been running TWS 10.26 successfully all day every day since 6 December. No problems at all. This includes autorestart every day. So I don't agree that there are problems with 10.26, at least as regards startup and auto-restart. Clearly there is a problem with your setup, but I don't think 10.26 is in any way the cause of it.

You say that 'it writes its logs to c:\jts, and not to c:\jts\1026', but it writes there because that is how it's configured in IBC. Let me explain: if you run TWS manually, ie without IBC, it does indeed write logs in C:\Jts\1026, because that's where it stores all its settings by default: and this includes logfiles. To be a bit more accurate, that's where it stores settings and launcher logs that are needed before login is completed, and per-user settings and logfiles are stored in the subfolder with a long filename (fcenjfkpcboicdnfienbgnkekhifhnllmnlhgcin in my case, and in yours it's cjmgpffcjehgdphobpeddjggomchmllnbpkogpfp.

But IBC enables you to configure a different location, which you specify with the TWS_SETTINGS_PATH variable in StartTWS.bat (please read and understand the notes on this in the .bat file, and the related but deprecated IbDir setting in config.ini). This setting defaults to the value of the TWS_PATH variable, which you have set to C:\Jts, so that's where your logfiles and settings are stored. (The default here is different from when you run TWS manually for good reasons, but I don't have time to justify it here.) For example, In my configuration I have folders called C:\Jts\SettingsLive and C:\Jts\SettingsPaper: this is because I prefer to keep anything to do with my live account well away from my paper account, to reduce the chance of damaging something due to a silly mistake (easy when you get to my age...). Similarly I have two IBC folders, one for live and one for paper.

I'm rather averse to you troubling IBKR with things like where TWS is writing logfiles without checking with me first: if it's an IBC configuration issue (or bug!), it will only confuse and annoy them, and waste their time. I'm always very keen that IBC should not impose any support burden on IBKR except for cases where it really is their problem (and there have been very few of them over the years).

Regarding what you plan to do, that's fine with me. Use IBC 3.18.0 for current TWS/Gateway versions.

@akatsev
Copy link
Author

akatsev commented Dec 30, 2023 via email

@rlktradewright
Copy link
Member

Can I make a suggestion? This is to hopefuly save you a lot of time and trouble trying to diagnose your problem, because I suspect I could do it much quicker than you.

What I'm suggesting is that you make a zip of your complete C:\Jts and C:\IBC drectory trees (preserving the directory structure of course). I can then replicate this exactly on one of my computers and take a forensic look at it, including running it against my account to see if I hit the same problem

You can of course remove old TWS versions and old logfiles from the zip (eg 10.19 and 10.25) to reduce the size. And you should remove your username and password from config.ini (make sure that's included as well).

Then put this zip file somewhere I can get at it (eg Dropbox or OneDrive, or even just put it on Github) and send me a link. If you prefer, you could email me the link directly to: info at aultan.com

@akatsev
Copy link
Author

akatsev commented Dec 30, 2023 via email

@akatsev
Copy link
Author

akatsev commented Dec 31, 2023 via email

@rlktradewright
Copy link
Member

Ok, thanks for that.

Well I've done what I can, which is basically run TWS 1026 with your setup, including the US/Eastern timezone. I couldnt use your TWS settings in the cjmgpffcjehgdphobpeddjggomchmllnbpkogpfp file, without having your login credentials, so I don't know whether there are any settings there that might be problematic. But that seems unlikely.

By the way, your cjmgpffcjehgdphobpeddjggomchmllnbpkogpfp folder didn't contain any of the xml settings files or log files that would normally be there: is this because you deleted them to reduce the size of the zip file, or are they really not there? In particular tws.xml contains all the settings that are managed by the TWS Global Configuration Editor: if that doesn't exist, then everything has its default settings.

Anyway, it all worked fine, including autorestart initiated via the File | Restart command.

I also couldn't test with the computer set to UTC time, because the computer I was running on is my domain controller and I'm not prepared to upset that in any way (I suspect that setting that would not cause any problems, but I don't want to risk it).

So, I'm afraid the conclusion that I've come to is that there is nothing wrong with your setup, and the problem with auto-restart is purely because it's using expired credentials.

There's really nothing more I can do until you wipe everything out and reconfigure it from scratch. But the most critical thing as far as auo-restart is concerned is to make sure you shut down TWS completely at some point during Sunday, so use the ClosedownAt setting in config.ini to force this.

@akatsev
Copy link
Author

akatsev commented Jan 1, 2024 via email

@akatsev
Copy link
Author

akatsev commented Jan 3, 2024 via email

@rlktradewright
Copy link
Member

If you want to discuss things like time settings in relation to TWS, please do so on the TWS API Group (https://groups.io/g/twsapi) or the IBC User Group (https://groups.io/g/ibcalpha), rather than here. I do not want to be your personal IT consultant!...

WHich you should use depends on whether your question is related to time in general, in connection with trading via the API or specifically time-related settings in IBC

Posting questions on those groups is likely to get a wider range of opinions and suggestions than just me, and of course the conversation can be potentially beneficial to many more users than here.

@akatsev
Copy link
Author

akatsev commented Jan 6, 2024 via email

@akatsev
Copy link
Author

akatsev commented Jan 11, 2024 via email

@rlktradewright
Copy link
Member

This is getting out of hand. You didn't attach the logfile, you quoted it inline which I specifically asked you not to do. Note only that, but it's not even the complete logfile, so it tells me nothing.

You say your other computer is ok, so please can you send me BOTH logfiles so I can compare them (attached, not quoted please). There has to be some difference!

Please also tell me what the AutoLogoff or AutoRestart setting in the Lock and Exit page of Global Configuration is set to.

@akatsev
Copy link
Author

akatsev commented Jan 11, 2024 via email

@rlktradewright
Copy link
Member

Here's the best way (in my opinion) to handle posting in GitHub, and attaching files. The basic rule is always to work directly on the GitHub website, not via email.

So when you're replying to a post that you've been notified via email, don't just reply to the email: click on the 'View it on GitHub' link at the end of the email. This will take you straight to the right page in GitHub, where you can type your response directly.

To attach a file, just click on the 'Paste, drop, or click to add files' link just below the textbox where you're typing your post, and select the file.

The only downside to this approach is that you'll have to login to GitHub (if you aren't already logged in).

Anyway, I'm glad you found the source of the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@rlktradewright @akatsev