[CDEC-439] Cleaning ntp client #3264
Conversation
networking/src/Ntp/Client.hs
Outdated
|
|
||
| let poll = ntpPollDelay (ncSettings cli) | ||
|
|
||
| _ <- withAsync |
There was a problem hiding this comment.
That looks like you are discarding the result of a computation. Since the last statement in the computation is cancel a whose result type is IO (), the above could be written as:
() <- withAsync
which is more obviously correct and hence easier for the reviewer.
There was a problem hiding this comment.
That's true. PureScript has a class for types which can be ignored without a warning (with an instance for ()), it would be nice to have it in Haskell.
networking/src/Ntp/Client.hs
Outdated
| ncStatus <- newTVarIO NtpSyncPending | ||
| -- using async so the ntp thread will be left running even if the parent | ||
| -- thread finished. | ||
| _ <- liftIO $ async (spawnNtpClient ntpSettings ncStatus) |
There was a problem hiding this comment.
How many times is this function likely to be called? Does spawnNtpClient ever terminate?
If it never terminates, how do we make sure we don't get hundreds, thousands or even more threads?
If if just runs, does it job and then terminates, it would be nice to have a comment to that effect.
There was a problem hiding this comment.
It just runs once, a comment should be in place indeed.
networking/src/Ntp/Client.hs
Outdated
| (<>) <$> (Option <$> createAndBindSock IPv4 addrs) | ||
| <*> (Option <$> createAndBindSock IPv6 addrs) | ||
|
|
||
| handlerE :: SomeException -> IO (Option Sockets) |
There was a problem hiding this comment.
We're using Control.Exception.Safe in this module so SomeException does not include async exxecptions like stack and heap overflow, right?
There was a problem hiding this comment.
Yes, it's a bit inconsistent though: Ntp.Client is using safe-exceptions while Ntp.Util is using Contorl.Exception (but it's never catching async exceptions, only IOException and one custom exception). I think we should just catch IOExceptions in Ntp.Client as well rather than SomeException.
coot
requested review from
adinapoli-iohk,
akegalj,
denisshevchenko and
parsonsmatt
as code owners
|
@erikd I updated with your comments with two additions:
|
Plus: remove safe-exception, and instead catch only `IOException`s.
coot
force-pushed
the
coot/cdec-439
branch
2 times, most recently
from
5bb9194 to
8e17a6f
Compare
networking/src/Ntp/Util.hs
Outdated
| = EBFirst !a | ||
| | EBSecond !b | ||
| | EBBoth !a !b | ||
| deriving (Show, Eq, Ord) |
There was a problem hiding this comment.
aka the these datatype, though this is strict
There was a problem hiding this comment.
Indeed, the strictness doesn't matter that much, since we are just holding max three values, so if you prefer we can pull these as a dependency. these hasn't been yet used anywhere in cardano-sl but it's a small dependency so I don't mind.
| {-# LANGUAGE TypeApplications #-} | ||
|
|
||
| module Test.NtpSpec | ||
| ( spec |
networking/test/Test/NtpSpec.hs
Outdated
| let offset = clockOffsetPure npoNtpPacket npoDestinationTime | ||
| in npoOffset === offset | ||
| describe "realMcsToNtp" $ do | ||
| it "should be right inverse of ntpToRealMcs" $ property $ \(NtpMicrosecond x) -> |
There was a problem hiding this comment.
hspec has a shorthand for this; it msg (property f) can be prop msg f
networking/test/Test/NtpSpec.hs
Outdated
| x === uncurry ntpToRealMcs (realMcsToNtp x) | ||
| it "should be left inverse of ntpToRealMcs" $ property $ \(NtpTime x@(sec, frac)) -> | ||
| let (sec', frac') = realMcsToNtp (uncurry ntpToRealMcs x) | ||
| in sec === sec' .&&. frac `div` 4310 === frac' `div` 4310 |
There was a problem hiding this comment.
4310 is a bit of a magic number here. What's the significance? Could it be named?
There was a problem hiding this comment.
It is 1000000 div 232, in the ntp spec time is represented in seconds + a fraction of a second which resolved to 232 picoseconds, so there are exactly 4310 of them in a milisecond.
| @@ -66,7 +75,7 @@ data NtpClientSettings = NtpClientSettings | |||
| data NtpClient = NtpClient | |||
| { ncSockets :: TVar Sockets | |||
| -- ^ Ntp client sockets: ipv4 / ipv6 / both. | |||
| , ncState :: TVar (Maybe [NtpOffset]) | |||
| , ncState :: TVar [NtpOffset] | |||
There was a problem hiding this comment.
Was there previously a semantic distinction between Nothing and Just []?
There was a problem hiding this comment.
There was, but it wasn't necessary. It was used to not accept request which came late. Every packet comes with information when it was sent so we can discard it based on that information. This is now done in clockOffset function
| (do | ||
| -- wait for responses and update status | ||
| _ <- timeout respTimeout waitForResponses | ||
| updateStatus cli |
There was a problem hiding this comment.
Could this part be done purely? Instead of having waitForResponses mutate some TVar, have it return the responses directly, then make updateStatus take it as an argument?
There was a problem hiding this comment.
Good point, I was actually thinking about doing this too. I haven't done it because we need ncState :: NtpClient -> TVar [NtpOffset] variable to communicate between the listener thread and sending thread. But we can change how the client is run: if we start the listener thread in this place we will not need this tvar. The communication will be done just using ncStatus.
There was a problem hiding this comment.
We need to listen on two sockets, so I think there's no way around impure ncState. Maybe you have some other idea?
There was a problem hiding this comment.
I wonder if having a receiving thread (startReceive) is the wrong way around. Do we ever expect to get a message without first sending one? That's to say, are we acting as an ntp server as well as a client? If not, maybe we should recvFrom the socket on the same thread after sending on it. That would give a simpler implementation, with less concurrency and fewer mutable cells.
There was a problem hiding this comment.
It is, but I haven't changed that. I just didn't want to rewrite everything. I will add a todo, and let's move on.
| handleIOException addressFamily e = do | ||
| logDebug $ sformat ("startReceive failed with reason: "%shown) e | ||
| threadDelay 5000000 | ||
| udpLocalAddresses >>= createAndBindSock addressFamily >>= \case |
There was a problem hiding this comment.
Could we somehow reuse/consolidate mkSockets from below?
There was a problem hiding this comment.
Giving this a second thought: in this place we are just recreating the socket that failed, while mkSockets creates both of them, i.e. IPv4 and IPv6; so I don't think we should reuse it here.
|
|
||
| instance Binary NtpPacket where | ||
| put NtpPacket{..} = do | ||
| putWord8 ntpParams | ||
| putWord8 0 | ||
| putInt8 ntpPoll | ||
| putWord8 0 |
There was a problem hiding this comment.
A change to the binary format? Is this some standardised NTP format?
There was a problem hiding this comment.
Not really, I added bits just so we have a test that put and get are inverse of each other.
There was a problem hiding this comment.
The added quickcheck test
There was a problem hiding this comment.
They weren't inverse prior to this change?
So I'm guessing the replicateM_ 4 $ putWord8 0 was there beecause ntpPoll was assumed to be 0. Here there's a putInt8 rather than putWord8. Same encoding?
Then there's the replicateM_ 9 $ PutWord32be 0, which is replaced by replicateM_ 5 $ putWord32be 0 but where are the remaining 4? ntpOriginTime ^.. each and ntpReceivedTime ^.. each give 2 Word32s each? Lens combinators are annoying hard to search for.
* use `These` from `these` package * `prop` in tests * comment why `4310` appears in the test
|
|
||
| instance Binary NtpPacket where | ||
| put NtpPacket{..} = do | ||
| putWord8 ntpParams | ||
| putWord8 0 | ||
| putInt8 ntpPoll | ||
| putWord8 0 |
There was a problem hiding this comment.
They weren't inverse prior to this change?
So I'm guessing the replicateM_ 4 $ putWord8 0 was there beecause ntpPoll was assumed to be 0. Here there's a putInt8 rather than putWord8. Same encoding?
Then there's the replicateM_ 9 $ PutWord32be 0, which is replaced by replicateM_ 5 $ putWord32be 0 but where are the remaining 4? ntpOriginTime ^.. each and ntpReceivedTime ^.. each give 2 Word32s each? Lens combinators are annoying hard to search for.
networking/src/Ntp/Packet.hs
Outdated
| let -- microseconds | ||
| secMicro :: Integer | ||
| secMicro = (fromIntegral sec - ntpTimestampDelta) * 1000000 | ||
| -- Each fraction resolves to 232 pico seconds (`232 ≈ 10000000/4294`) |
There was a problem hiding this comment.
There's an extra 0 in the numerator.
The explanation could be better. We divide 1 second into 2 ^ 32 parts, giving 2.3283064365386963e-10 as the quantum. A picosecond is 10e-12 of a second, so this is 232 picoseconds.
There was a problem hiding this comment.
It wasn't. Though previously Int was used as type for poll in NtpPacket we put 0 :: Word8 when encoding it (because it does not matter). I changed Int to Int8, the spec sais that:
Some time values are represented in exponent format, including the
precision, time constant, and poll interval. These are in 8-bit
signed integer format in log2 (log base 2) seconds.
So I Int8 should be used. Before it was Int. We ignore this field, but let's follow the spec.
But put and get weren't inverses because when we were encoding a packet we always put 0 in the origin and received time. This also does not matter for us, since we're just a client, but why not follow the spec if it does not cost much.
Indeed each the result of realMcsToNtp return a pair of Word32 (seconds since 1 Jan 1900 and fractions).
There was a problem hiding this comment.
Sorry, I put this comment under wrong thread.
…hk/coot/cdec-439 [CDEC-439] Cleaning ntp client
Description
I cleaned the ntp client a bit.
The ntp client resolves addresses and takes first ipv4 and ipv6 addresses. It now will also try the other address in case of failure (it may happen that ipv4/6 is not supported by the os).
And I added tests:
Linked issue
https://iohk.myjetbrains.com/youtrack/issue/CDEC-439
Type of change
Developer checklist
Testing checklist
QA Steps
Screenshots (if available)