config: command to apply profile after init #4195

magik6k · 2017-09-01T01:48:42Z

TODO:

Revert command
Tests
Docs

whyrusleeping · 2017-09-01T21:28:03Z

It would be nice to print out what changes are being made

whyrusleeping · 2017-09-21T21:44:04Z

core/commands/config.go

+	Run: func(req cmds.Request, res cmds.Response) {
+		args := req.Arguments()
+
+		profile, ok := config.Profiles[args[0]]


instead of binding it to args and just using the [0] on it, why not do:

profilename := req.Arguments()[0]

whyrusleeping · 2017-09-21T21:45:58Z

repo/config/profile.go

+				"/ip4/203.0.113.0/ipcidr/24",
+				"/ip4/240.0.0.0/ipcidr/4",
+			}
+


Not really related to this PR, but i was just thinking that we should probably add LAN addresses to the NoAnnounce field for the server config.

whyrusleeping · 2017-09-22T04:44:05Z

core/commands/config.go

+
+		profile, ok := config.Profiles[req.Arguments()[0]]
+		if !ok {
+			res.SetError(fmt.Errorf("%s in not a profile", req.Arguments()[0]), cmds.ErrNormal)


whyrusleeping · 2017-09-22T04:44:56Z

core/commands/config.go

+}
+
+func transformConfig(configRoot string, transformer config.Transformer) error {
+	r, err := fsrepo.Open(configRoot)


if the daemon is running while the user runs this they might run into issues. You cant open the repo again

probably best to pass in the repo from the IpfsNode

That's what other commands here do. Looking at repo.Open it should be safe:

func Open(repoPath string) (repo.Repo, error) { fn := func() (repo.Repo, error) { return open(repoPath) } return onlyOne.Open(repoPath, fn) }

oh, i see. good point.

whyrusleeping · 2017-10-28T21:42:45Z

docs/config.md

@@ -267,3 +268,17 @@ Disables the p2p-circuit relay transport.
 - `EnableRelayHop`
 Enables HOP relay for the node. If this is enabled, the node will act as
 an intermediate (Hop Relay) node in relay circuits for connected peers.
+
+## Profiles


Hrm... i'm not sure this belongs here. Maybe put it at the top of the document? to distinguish it from just looking like another config field

Moved it to the header, makes a bit more sense now

magik6k · 2017-11-01T16:30:20Z

@whyrusleeping Is there anything else to do here? Getting this in would make #4154 a bit easier for me to test.

whyrusleeping · 2017-11-04T19:00:35Z

repo/config/profile.go

+			c.Discovery.MDNS.Enabled = false
+			return nil
+		},
+		Unapply: func(c *Config) error {


Maybe call it Revert instead of Unapply?

whyrusleeping · 2017-11-04T19:02:17Z

One small comment, then LGTM

kevina

See individual comments.

Also could use some some ore comprehensive tests.

kevina · 2017-11-20T19:30:08Z

core/commands/config.go

+var configProfileRevertCmd = &cmds.Command{
+	Helptext: cmds.HelpText{
+		Tagline: "Revert profile changes.",
+		ShortDescription: `Reverts profile-related changes to the config.


This should be Reverts profile-related changes to the default config..

kevina · 2017-11-20T19:36:03Z

docs/config.md

+Reduces external interference, useful for running ipfs in test environments.
+Note that with these settings node won't be able to talk to the rest of the
+network without manual bootstrap.
+
 ## Table of Contents


@whyrusleeping should we document the badgerds profile here but label it as experimental?

Its somewhat out of scope, but probably a good idea.

Out of scope?

I meant to add under the new Profiles section in case that wasn't clear.

@kevina ah, gotcha. Yeah, lets get that in here.

kevina · 2017-11-20T19:39:58Z

I also like we should create backup files automatically. We could call them: .config.preapply-PROFILE and .config.prerevert.PROFILE

@whyrusleeping others, thoughs?

whyrusleeping · 2017-11-21T00:39:52Z

@kevina I do like the idea of making backups automatically. That will require some changes to the repo abstraction I think.

magik6k · 2017-11-25T02:53:45Z

Rebased, updated to use new commands
Did what @kevina suggested in the review
Backups are todo, other than that that PR is mergable

License: MIT Signed-off-by: Łukasz Magiera <[email protected]>

magik6k · 2017-12-15T21:19:22Z

Backups done

whyrusleeping · 2017-12-15T23:51:01Z

repo/mock.go

@@ -28,6 +28,10 @@ func (m *Mock) SetConfig(updated *config.Config) error {
 	return nil
 }

+func (m *Mock) BackupConfig(prefix string) (string, error) {
+	return "config-" + prefix + "-backup", nil


maybe this can just return errTODO like the other methods? If we arent actually going to back up the config, we probably should bother to pretend with the name

whyrusleeping

The backups look pretty cleanly done, I just have one comment on the mock repo.

License: MIT Signed-off-by: Łukasz Magiera <[email protected]>

frrist · 2017-12-16T00:37:06Z

docs/config.md

+  If you apply this profile after `ipfs init`, you will need to convert your
+  datastore to the new configuration. You can do this using [ipfs-ds-convert](https://github.com/ipfs/ipfs-ds-convert)
+
+  WARNING: badger datastore is experimantal. Make sure to backup your data


badger datastore is experimental

Stebalien · 2017-12-16T00:43:50Z

docs/config.md

+- `server`
+
+  Recommended for nodes with public IPv4 address, disables host and content
+  discovery in local networks.


Or those running on VPSs.

Stebalien · 2017-12-16T00:49:15Z

core/commands/config.go

+
+	Subcommands: map[string]*cmds.Command{
+		"apply":  configProfileApplyCmd,
+		"revert": configProfileRevertCmd,


Actually reverting is impossible (for us). We're not reverting, we're applying the inverse of the profile. Do we really need to provide this feature?

We could, instead, provide inverse profiles: local-discovery and flatfs-ds.

I'm worried that a user will think that profiles act like drop-ins (that's how a lot of unix daemons work) and will believe that revert will revert the apply.

Alternatively, we could just do this with layered drop-ins (but that's more complex).

I agree with @Stebalien this is not really a revert so we perhaps we should name it something else, not sure what, "unapply" comes to mind.

The idea of instead providing inverse profiles is also a good idea.

I think I like the idea of inverse profiles better, I'll see how that looks

Stebalien · 2017-12-16T00:49:48Z

repo/config/profile.go

+				"/ip4/240.0.0.0/ipcidr/4",
+			}
+
+			c.Addresses.NoAnnounce = append(c.Addresses.NoAnnounce, defaultServerFilters...)


We should probably try merging these instead of just appending.

Stebalien · 2017-12-16T00:50:03Z

repo/config/profile.go

+			return nil
+		},
+		Revert: func(c *Config) error {
+			c.Addresses.NoAnnounce = []string{}


If we do keep the revert command, we should only remove the ones we've added.

kevina

I would use more helpful names for the backup. The name config-profile-444132629 is not very descriptive. This also makes it hard to find the right config backup to undo the change.

I made suggestions for improving on this. However, I am not set on this, so I can change my review if no one else agrees with me.

If I was writing this I would also use a timestamp rather then TempFile so that if there are multiple backups it is clear what the latest one is. Another option would be to just overright the old backup, that is what most other programs would do.

kevina · 2017-12-16T00:54:55Z

repo/config/profile.go

-		c.Discovery.MDNS.Enabled = false
-		return nil
+// Transformer is a function which takes configuration and applies some filter to it
+type Transformer func(c *Config) error


I would make this a struct something like

type Transformer struct { Name string Apply func(c *Config) error }

kevina · 2017-12-16T00:56:52Z

core/commands/config.go

+		return err
+	}
+
+	_, err = r.BackupConfig("profile-")


After making config.Transformer I would make this

_, err = r.BackupConfig(transformer.Name + "-")

To provide more helpful backup names.

License: MIT Signed-off-by: Łukasz Magiera <[email protected]>

magik6k · 2017-12-16T18:55:34Z

Refactored revert into inverse profiles, cc @Stebalien

kevina · 2017-12-16T19:18:20Z

test/sharness/t0021-config.sh

+    ipfs config profile apply '${inverse_profile}'
+  '
+
+  test_expect_success "config is back to previous state after ${profile} revert" '


This test is a bit misnamed now.

kevina

Some more nitpicks on how backup are handled.

Otherwise looks good.

kevina · 2017-12-17T09:33:33Z

docs/config.md

+#### Profiles
+Configuration profiles allow to tweak configuration quickly. Profiles can be
+applied with `--profile` flag to `ipfs init` or with `ipfs config profile apply`
+command.


I would mention that a backup is created and what it is called.

kevina · 2017-12-17T09:35:26Z

core/commands/config.go

+	},
+}
+
+func transformConfig(configRoot string, backupName string, transformer config.Transformer) error {


I would call backupName configName.

kevina · 2017-12-17T09:37:50Z

core/commands/config.go

+		return err
+	}
+
+	_, err = r.BackupConfig(backupName + "-")


Nit. I would make this "pre-" + configName + "-" to make it clear that the backup before the config was applied.

License: MIT Signed-off-by: Łukasz Magiera <[email protected]>

whyrusleeping · 2018-01-02T22:02:21Z

Finally getting this one in, Thanks @magik6k !

magik6k force-pushed the feat/config-patch branch 2 times, most recently from 865f4fb to 128e6ba Compare September 12, 2017 23:44

whyrusleeping reviewed Sep 21, 2017

View reviewed changes

magik6k force-pushed the feat/config-patch branch 3 times, most recently from a017c41 to 48f1069 Compare September 21, 2017 22:26

whyrusleeping reviewed Sep 22, 2017

View reviewed changes

magik6k force-pushed the feat/config-patch branch 3 times, most recently from eeaaceb to ca5e92f Compare September 22, 2017 09:29

magik6k force-pushed the feat/config-patch branch from ca5e92f to a956439 Compare October 6, 2017 17:47

magik6k mentioned this pull request Oct 26, 2017

Running IPFS on VPS providers triggers netscan detection, risking account termination #4343

Closed

whyrusleeping reviewed Oct 28, 2017

View reviewed changes

magik6k force-pushed the feat/config-patch branch from a956439 to c875c01 Compare October 29, 2017 19:54

ghost assigned magik6k Oct 29, 2017

ghost added the status/in-progress In progress label Oct 29, 2017

whyrusleeping reviewed Nov 4, 2017

View reviewed changes

whyrusleeping requested review from Kubuxu and kevina November 19, 2017 02:17

kevina suggested changes Nov 20, 2017

View reviewed changes

magik6k force-pushed the feat/config-patch branch 2 times, most recently from 248e971 to 81579d6 Compare November 25, 2017 02:48

magik6k added 2 commits December 15, 2017 15:37

config-patch: update to new commands lib

c573d3d

License: MIT Signed-off-by: Łukasz Magiera <[email protected]>

config-patch: apply review suggestions

2514c74

License: MIT Signed-off-by: Łukasz Magiera <[email protected]>

magik6k force-pushed the feat/config-patch branch 2 times, most recently from 917f232 to bebdd2b Compare December 15, 2017 20:43

kevina self-requested a review December 15, 2017 21:22

whyrusleeping reviewed Dec 15, 2017

View reviewed changes

whyrusleeping approved these changes Dec 15, 2017

View reviewed changes

whyrusleeping requested review from Stebalien and frrist December 15, 2017 23:52

config-patch: backup config

0ff9b24

License: MIT Signed-off-by: Łukasz Magiera <[email protected]>

magik6k force-pushed the feat/config-patch branch from bebdd2b to 0ff9b24 Compare December 15, 2017 23:59

frrist reviewed Dec 16, 2017

View reviewed changes

frrist approved these changes Dec 16, 2017

View reviewed changes

Stebalien reviewed Dec 16, 2017

View reviewed changes

kevina suggested changes Dec 16, 2017

View reviewed changes

config-patch: docs typo, fix server profile

acb4edc

License: MIT Signed-off-by: Łukasz Magiera <[email protected]>

kevina reviewed Dec 16, 2017

View reviewed changes

magik6k force-pushed the feat/config-patch branch 2 times, most recently from c19cc7f to 0546f1f Compare December 16, 2017 19:54

Stebalien approved these changes Dec 16, 2017

View reviewed changes

kevina suggested changes Dec 17, 2017

View reviewed changes

config-patch: Inverse profiles

ac26cf1

License: MIT Signed-off-by: Łukasz Magiera <[email protected]>

magik6k force-pushed the feat/config-patch branch from 0546f1f to ac26cf1 Compare December 17, 2017 14:29

whyrusleeping approved these changes Jan 2, 2018

View reviewed changes

whyrusleeping merged commit f4fd369 into master Jan 2, 2018

whyrusleeping deleted the feat/config-patch branch January 2, 2018 22:02

ghost removed the status/in-progress In progress label Jan 2, 2018

config: command to apply profile after init #4195

config: command to apply profile after init #4195

Conversation

magik6k commented Sep 1, 2017 • edited Loading

whyrusleeping commented Sep 1, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

magik6k commented Nov 1, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

whyrusleeping commented Nov 4, 2017

kevina left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kevina commented Nov 20, 2017

whyrusleeping commented Nov 21, 2017

magik6k commented Nov 25, 2017

magik6k commented Dec 15, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

whyrusleeping left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Stebalien Dec 16, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kevina left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

magik6k commented Dec 16, 2017

Choose a reason for hiding this comment

kevina left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

whyrusleeping commented Jan 2, 2018

magik6k commented Sep 1, 2017 •

edited

Loading

Stebalien Dec 16, 2017 •

edited

Loading