"Secrets" not working

[candlerb]

Netbox 66a16dd under Ubuntu 14.04, built according to getting-started.md

I tried to create a secret for a device. It told me I needed to create a user key and navigated me to that place. So I generated a user key within the browser, and copy-pasted the private key to a local file.

I then tried again to add a secret to a device. It prompted me to paste my private key into the web interface - I did so.

But from then on, attempting to add a secret to a device just says:

"Invalid RSA key. Please ensure that your key is in PEM (base64) format."

I have tried:

• Logging out and logging back in
• Quitting and restarting the browser (Chrome under OSX)
• Regenerating and re-activating my user key

I have never been prompted to paste my private key in again, but I still get the "Invalid RSA key" message. Inspecting the HTML page shows that the private key is in a hidden form variable. I'm not sure how this is being kept "sticky", but at very least logging out should clear it and require it to be re-entered when required.

Side issue: I would be a lot happier if this used GPG keys, not RSA keys. GPG is robustly-designed, whereas it's very easy to make an insecure system from low-level crypto primitives. I'm particularly unhappy about private keys being constantly pasted back-and-forth between web client and server.

GPG allows the same document to be encrypted for multiple recipients: effectively there is no global "master key" but rather each document contains its own session key, encrypted with multiple public keys.

This in turn means that it would be unnecessary to provide your private key into the web interface when adding a new device secret.

[ptman]

In what format was your RSA key? According to http://fileformats.archiveteam.org/wiki/PEM_encoded_RSA_private_key it should be a plain text file with:

-----BEGIN RSA PRIVATE KEY-----
<...base64-encoded key...>
-----END RSA PRIVATE KEY-----

I agree with you that pasting a private key into the browser sounds like a less than fully thought out system. Have you looked at other team password storage systems? Such as rattic, webpasswordsafe or some commercial offerings? Can access be managed per user or group?

[candlerb]

In what format was your RSA key?

Exactly the format that the Netbox web interface generated it :-)

Actually I did originally try pasting in a GPG public key, but it was rejected; so I read the docs and then used the web interface to generate the public and private key pair. It has happily saved the public one in the database, and the private key is indeed in the format you showed.

I agree with you that pasting a private key into the browser sounds like a less than fully thought out system. Have you looked at other team password storage systems?

Yep, passwordstore is a sound design with one GPG file per password in a directory tree; and GPG allows the same file to be encrypted to multiple recipients.

I am thinking that what we really want here is not just documentation of passwords, but the ability to use those passwords by other backend systems driven from Netbox (e.g. to ssh into a router, or make an SNMP query, or to connect to IPMI). GPG would make this straightforward and secure.

[candlerb]

Update: after leaving my terminal for a bit to get lunch, and then trying to add a secret to a device, I got prompted for the private key, yay! (Some cookie has expired?)

So I pasted it in. Unfortunately I can now see that I pasted in had a line truncated; the result was "Invalid key detected: Incorrect padding".

I do have the correct key available, but no way to tell Netbox to ignore the bad key I gave it before!

I guess I'll have to leave it for a while to expire again. I tried changing SECRET_KEY and this forcibly logged me out, but it's still using the same cached private key.

[candlerb]

Switched to a different browser (Firefox). That gave me another opportunity to paste in my private key, correctly this time.

Secret is created successfully - displays as asterisks.

<div class="col-md-8" id="secret_1">********</div>

However when I click the Unlock button it just changes to empty string

<div class="col-md-8" id="secret_1"></div>

Now that I was able to create the secret in Firefox, returning to Chrome and displaying the Device did prompt me again for my key when I clicked Unlock. But again it displayed an empty secret.

So to summarise problems:

• Insufficient validation of private key when uploading it
• If private key is found to be unusable when encrypting, there is no opportunity to switch it for another key (probably should just forget the cached key as soon as there's an error)
• Unable to display decrypted secret

[jeremystretch]

What is the response from the server when you click the "unlock" button? You can see this using the inspection tool (F12) in Firefox and selecting the Network tab. You should see a POST to /api/secrets/secrets/{pk}/.

[candlerb]

Here's tcpdump output

        0x00f0:  0d0a 0d0a 7b22 6964 223a 312c 2264 6576  ....{"id":1,"dev
        0x0100:  6963 6522 3a7b 2269 6422 3a31 2c22 6e61  ice":{"id":1,"na
        0x0110:  6d65 223a 2254 6573 7420 726f 7574 6572  me":"Test.router
        0x0120:  222c 2270 7269 6d61 7279 5f69 7022 3a7b  ","primary_ip":{
        0x0130:  2269 6422 3a31 2c22 6661 6d69 6c79 223a  "id":1,"family":
        0x0140:  342c 2261 6464 7265 7373 223a 2231 2e32  4,"address":"1.2
        0x0150:  2e33 2e34 2f32 3422 7d7d 2c22 726f 6c65  .3.4/24"}},"role
        0x0160:  223a 7b22 6964 223a 312c 226e 616d 6522  ":{"id":1,"name"
        0x0170:  3a22 534e 4d50 2063 6f6d 6d75 6e69 7479  :"SNMP.community
        0x0180:  222c 2273 6c75 6722 3a22 736e 6d70 2d63  ","slug":"snmp-c
        0x0190:  6f6d 6d75 6e69 7479 227d 2c22 6e61 6d65  ommunity"},"name
        0x01a0:  223a 2222 2c22 706c 6169 6e74 6578 7422  ":"","plaintext"
        0x01b0:  3a6e 756c 6c2c 2268 6173 6822 3a22 7062  :null,"hash":"pb
        0x01c0:  6b64 6632 5f73 6861 3235 3624 3130 3030  kdf2_sha256$1000
        0x01d0:  246c 4b53 7835 3567 5368 596f 3024 4a55  $lKSx55gShYo0$JU
        0x01e0:  4369 3531 4536 7a42 7551 4366 7279 4a73  Ci51E6zBuQCfryJs
        0x01f0:  6449 6638 424b 6e44 5161 6930 7347 4657  dIf8BKnDQai0sGFW
        0x0200:  7346 5235 6246 7a75 593d 222c 2263 7265  sFR5bFzuY=","cre
        0x0210:  6174 6564 223a 2232 3031 362d 3036 2d32  ated":"2016-06-2
        0x0220:  3822 2c22 6c61 7374 5f75 7064 6174 6564  8","last_updated
        0x0230:  223a 2232 3031 362d 3036 2d32 3854 3134  ":"2016-06-28T14
        0x0240:  3a34 303a 3132 2e38 3835 3934 395a 227d  :40:12.885949Z"}

I note "plaintext": null

[mhmsh]

Same issue here.

[jeremystretch]

Looks like your key isn't being included with the request. Could you please provide a capture of the request and response to confirm this? You can use the web developer toolkit in Firefox or Chrome to capture the request.

[candlerb]

(Using Chrome). The key is definitely being included in the request.

Headers

General

Request URL:http://100.64.2.5:8000/api/secrets/secrets/1/
Request Method:POST
Status Code:200 OK
Remote Address:100.64.2.5:8000

Response headers

HTTP/1.0 200 OK
Date: Thu, 30 Jun 2016 19:34:52 GMT
Server: WSGIServer/0.1 Python/2.7.6
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN
Content-Type: application/json
Allow: GET, POST, HEAD, OPTIONS

Request headers

POST /api/secrets/secrets/1/ HTTP/1.1
Host: 100.64.2.5:8000
Connection: keep-alive
Content-Length: 1844
Origin: http://100.64.2.5:8000
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Accept: application/json, text/javascript, */*; q=0.01
X-Requested-With: XMLHttpRequest
X-CSRFToken: GrBj0fPwYKQ4JIKSo1IfKKAuQvYedByt
DNT: 1
Referer: http://100.64.2.5:8000/dcim/devices/1/
Accept-Encoding: gzip, deflate
Accept-Language: en-GB,en;q=0.8,en-US;q=0.6
Cookie: sessionid=gmr9768shi4dsw4633fxic2r0edbcb6m; csrftoken=GrBj0fPwYKQ4JIKSo1IfKKAuQvYedByt

Form data

View source:

private_key=-----BEGIN+RSA+PRIVATE+KEY-----%0AMIIEpQIBAAKCAQEAqALemmFZJdJfhmGR5fHfPz4i9QqgVOnwd261fZtuvC7dHT%2FK%0ALuy8SaZSXYvfIAGja7%2Fp1BVo7isKj%2F%2BT7B7JnGxO2z4kVc5eMInIyDVyNq%2BbLvEa%0AWfdsQt6SeRIX%2FjV3PQrWhGH9bvIsqTpTD2i8GFcd4yUGTBfTrOSNo9tnRs0JcPQE%0A9bF7GNFHWjQx3BlUA6sUfZN%2F0%2Fz9c4cZdGkjHbYhkKV3vyEKP4DQHGdFN8Esg8P%2F%0A3P32aUB4atVnH%2BWnulroRurkMmLo5J7CxT7PSPLlvF5cP4PSkJdzWSPWZEArUnd8%0An3Ztq2I1EKdzGmmWUr%2FvopnZGqtbqtSyVCoL4QIDAQABAoIBABfAeBnZI1I%2BgHSt%0A%2FINGg2ocbbK4stvuSqmHxC6it6%2Fw7B%2BDciDaJPBAjCpfaMbMlH7aHcN212O8P%2F0F%0Ag7MA0z3C7g7%2BpQJgb15rTDBJpgDnKIglW7VWTj7R2mE1VqEAi%2F%2FcNWF3vO1fXUL7%0AVNy70MOcJUSEanR9ddZeZUvB68LNZPsm9MCG1K43HYq1P9cjb1TxR5IncnKX7FSc%0AnsQmTUCdtg%2FZuToK4uA%2BgwdMoRkCOVo7XIq%2F91Kp2BECv6JS%2B08BtNyn8S8qe8mC%0A2yMjhr0yYmOV1k3o1HJDeAEI3njQn3Kae99r6k%2FTrv%2FhFZl4kxH0GW7nMSwUPRx2%0AouHzSjECgYEAwQloeRNFv%2FEQ%2BPlvKpnUSMMmYKUrCTkEpukWnJGaaKY2rcRE9J7C%0Ae%2FN1PydrCmtHENoCMpMcDOH88WtALq7dAXRpTabDnrq5RqA96ez22vqSTkiroT0v%0An%2BZ6Y%2FSB9k04QXpDejE304%2FiWdloNcR8oDg2thMb%2Fue1odDi2acGnH0CgYEA3s%2FT%0AY6sPYgthQz9ltt61RH4lXKFd61g7NPRZkkdTpj5GfIOhf3U%2FjmdqqgX82q5SpCWi%0AVis5zdMpQgITJOeNnpCE4VKujncXcLKNbOQk3dKBNR1RXMRnh5lmMQ%2BrwNIPx%2B9q%0APFj5gSnzVJw8TCh30bL%2FnMhr6vaob%2Btb8HpZHjUCgYEArqRwXHnPNORIUrh4lrYt%0AlNbZtb44vVNUnh1rU6N3aw5VkchsClRH6BRPEx%2F6e6aq2Ngj%2FNWTHySlz1UU8CdL%0A07EaRCKIAOc2EGuRGD5G576i9yhP%2FsR4fdXrDNt9aPRb3QFcYwR8WW9nfZpv6ZUm%0Am13AQeHwBiJj9U4irn%2BqP5ECgYEAnM%2FPipndnbc8oMOBpo5fLPFLAkJfgz75jG9c%0AiuLAMIQrv340ryupq2OFYlwMdSuy5evGsLaDXCR9KQOkzZhZx3sWyi4SxBl02dAQ%0AUM7QU33Pib40gQGJi5mennDVBEPS73fEYOyNzSXlJ2s91Ku2iFNeKs2zhg5KxVnz%0Avu%2BcwPECgYEAm%2BPV4vlvqf2Rv2CxsJmbJNQgJAiDva6OBiwtICwe6IkN7Z%2BJ9JTX%0AiSUFcvAJSnGvoEl8%2FwxVeOoJQvbj4k5R9n8PXblCWtx5nwHihvdYSxhZrWqWJjqo%0A0L4eC6Hq%2F%2BoGZYAkNXqkvou7Yk5OnX9Ipnk8oKiyES%2BiZ1atVsS00Ik%3D%0A-----END+RSA+PRIVATE+KEY-----

View parsed:

private_key:-----BEGIN RSA PRIVATE KEY-----
MIIEpQIBAAKCAQEAqALemmFZJdJfhmGR5fHfPz4i9QqgVOnwd261fZtuvC7dHT/K
Luy8SaZSXYvfIAGja7/p1BVo7isKj/+T7B7JnGxO2z4kVc5eMInIyDVyNq+bLvEa
WfdsQt6SeRIX/jV3PQrWhGH9bvIsqTpTD2i8GFcd4yUGTBfTrOSNo9tnRs0JcPQE
9bF7GNFHWjQx3BlUA6sUfZN/0/z9c4cZdGkjHbYhkKV3vyEKP4DQHGdFN8Esg8P/
3P32aUB4atVnH+WnulroRurkMmLo5J7CxT7PSPLlvF5cP4PSkJdzWSPWZEArUnd8
n3Ztq2I1EKdzGmmWUr/vopnZGqtbqtSyVCoL4QIDAQABAoIBABfAeBnZI1I+gHSt
/INGg2ocbbK4stvuSqmHxC6it6/w7B+DciDaJPBAjCpfaMbMlH7aHcN212O8P/0F
g7MA0z3C7g7+pQJgb15rTDBJpgDnKIglW7VWTj7R2mE1VqEAi//cNWF3vO1fXUL7
VNy70MOcJUSEanR9ddZeZUvB68LNZPsm9MCG1K43HYq1P9cjb1TxR5IncnKX7FSc
nsQmTUCdtg/ZuToK4uA+gwdMoRkCOVo7XIq/91Kp2BECv6JS+08BtNyn8S8qe8mC
2yMjhr0yYmOV1k3o1HJDeAEI3njQn3Kae99r6k/Trv/hFZl4kxH0GW7nMSwUPRx2
ouHzSjECgYEAwQloeRNFv/EQ+PlvKpnUSMMmYKUrCTkEpukWnJGaaKY2rcRE9J7C
e/N1PydrCmtHENoCMpMcDOH88WtALq7dAXRpTabDnrq5RqA96ez22vqSTkiroT0v
n+Z6Y/SB9k04QXpDejE304/iWdloNcR8oDg2thMb/ue1odDi2acGnH0CgYEA3s/T
Y6sPYgthQz9ltt61RH4lXKFd61g7NPRZkkdTpj5GfIOhf3U/jmdqqgX82q5SpCWi
Vis5zdMpQgITJOeNnpCE4VKujncXcLKNbOQk3dKBNR1RXMRnh5lmMQ+rwNIPx+9q
PFj5gSnzVJw8TCh30bL/nMhr6vaob+tb8HpZHjUCgYEArqRwXHnPNORIUrh4lrYt
lNbZtb44vVNUnh1rU6N3aw5VkchsClRH6BRPEx/6e6aq2Ngj/NWTHySlz1UU8CdL
07EaRCKIAOc2EGuRGD5G576i9yhP/sR4fdXrDNt9aPRb3QFcYwR8WW9nfZpv6ZUm
m13AQeHwBiJj9U4irn+qP5ECgYEAnM/Pipndnbc8oMOBpo5fLPFLAkJfgz75jG9c
iuLAMIQrv340ryupq2OFYlwMdSuy5evGsLaDXCR9KQOkzZhZx3sWyi4SxBl02dAQ
UM7QU33Pib40gQGJi5mennDVBEPS73fEYOyNzSXlJ2s91Ku2iFNeKs2zhg5KxVnz
vu+cwPECgYEAm+PV4vlvqf2Rv2CxsJmbJNQgJAiDva6OBiwtICwe6IkN7Z+J9JTX
iSUFcvAJSnGvoEl8/wxVeOoJQvbj4k5R9n8PXblCWtx5nwHihvdYSxhZrWqWJjqo
0L4eC6Hq/+oGZYAkNXqkvou7Yk5OnX9Ipnk8oKiyES+iZ1atVsS00Ik=
-----END RSA PRIVATE KEY-----

Response

{"id":1,"device":{"id":1,"name":"Test router","primary_ip":{"id":1,"family":4,"address":"1.2.3.4/24"}},"role":{"id":1,"name":"SNMP community","slug":"snmp-community"},"name":"","plaintext":null,"hash":"pbkdf2_sha256$1000$lKSx55gShYo0$JUCi51E6zBuQCfryJsdIf8BKnDQai0sGFWsFR5bFzuY=","created":"2016-06-28","last_updated":"2016-06-28T14:40:12.885949Z"}

[jeremystretch]

Sorry for the delay on this. I believe this is a permissions issue. Secrets are only decryptable by users who are permitted access by SecretRole (either individually or by group) and who have an active UserKey. Unfortunately, there is no error message in place to indicate the lack of permission.

If you edit the secret role in the admin UI under secrets > secret roles, you can assign your user (or a group of which your user is a member) to the role. This should grant you permission to decrypt the secret. Please let me know if that works.

[mhmsh]

Yep, that did it. Its working now.

[candlerb]

Thank you.

I am logged in as the "admin" user. In the django admin it's not entirely clear whether the admin user is enabled or not for this role:

But after selecting it and saving, it changes so that the user is highlighted in grey:

And it works after this.

So useful changes would be:

• When creating a new secret role in the main GUI, prompt for which users/groups should have access to that role. Or at least link to the django admin page saying that you need to add users/groups.
• When unlocking a secret, if the user does not have rights then generate a message saying so. And/or: grey out the unlock button beforehand, with a tooltip explaining why.
• Prevent creating a secret if you don't have rights to that secret role. Otherwise it's very confusing that you can create a secret when you do not have rights to unlock that same secret!

In fact, I'm surprised you can create a secret without being able to read it, because creating a secret implies you have access to the master key. If access control is managed at the level of secret roles, shouldn't each role have a separate key?

[jeremystretch]

Further improvements: edde021 granted all superusers permission to decrypt all secrets, and c19124f adds a note to the documentation.

[jeremystretch]

Prevent creating a secret if you don't have rights to that secret role. Otherwise it's very confusing that you can create a secret when you do not have rights to unlock that same secret!

There may be instances where you want a user to be able to create secrets but no decrypt them, for example a provisioning system that generates RADIUS keys.

creating a secret implies you have access to the master key.

Just to clarify, end users never have direct access to the master key.

shouldn't each role have a separate key?

This might be ideal, but may lead to performance problems. Deriving the master key is a (relatively) slow asymmetric decryption process, whereas decrypting individual secrets is very fast.

In the current scheme, only one key needs to be derived to enable decryption of all secrets. If we kept a separate key for each role, we'd need to decrypt a key for each role in the list of secrets to be decrypted. This could lead to long delays when decrypting secrets belonging to many different roles.

It may be worth taking another look at the permissions around secrets, but we've breached the original topic of this issue, so I'm going to mark it as closed.

[bogdanstoica35]

In my case it is not working. I have created a group with all the permission and added my user to that group. I have added both my user and the group to the secret roles but I am still getting this error: Invalid private key! Unable to encrypt secret data.

I am using the latest netbox version cloned from github.

Any clues?!

[eronlloyd]

@bogdanstoica35: I added all the Secrets permissions to a group and then assigning it to my user, it worked. I'm on 1.6.3. HOWEVER, while I was able to encrypt, but now I'm getting an error trying to decrypt. So I'm still having issues.

UPDATE: After just adding a few more, now it's working properly and I can view the secrets. Seems like there's still some reliability testing needed between storing the private key and the UI to view the data?

[bogdanstoica35]

It is working now, but only after upgrading to the latest version from github. Also, previously it was only working on Firefox. Now it works both on Chrome and Firefox.