feat: Privacy Preserving Learning

[manavsinghal157]

Part of the Empirical Analysis of Privacy Preserving Learning Project.

This PR introduces a command line argument that implements aggregated learning by saving only those features that have seen a minimum threshold of users thus upholding the privacy of the user.

Methodology:

• For each feature, a 32-bit vector is defined. (vowpalwabbit/array_parameters.h and vowpalwabbit/array_parameters_dense.h)
• We calculate a 5-bit hash of the tag of the example. (vowpalwabbit/parser.cc)
• For each feature weight updated by a non-zero value, we use the 5-bit hash to look up a bit in the 32-bit vector and set it to 1.(vowpalwabbit/gd_predict.h -> (vowpalwabbit/array_parameters.h and vowpalwabbit/array_parameters_dense.h))
• When saving the weights into a file, we calculate the number of bits set to 1 for a feature. If it is greater than the threshold, the weights for that feature are saved. (vowpalwabbit/gd.cc->(vowpalwabbit/array_parameters.h and vowpalwabbit/array_parameters_dense.h))

(The default value of the threshold is 10)

This PR includes:

• Command line argument to activate privacy preservation and set the threshold. (vowpalwabbit/parse_args.cc)
• Runtests to test the desired output on a small dataset. (test/core.vwtest.json)
• Unit-tests for checking output when threshold is reached for a feature and when it is not. (test/unit_test/weights_test.cc)
• Benchmarks to test time taken for learning in privacy preserving method. (test/benchmarks/standalone/benchmark_text_input.cc )

Implementation details:

--privacy_activation : To activate the feature
--privacy_activation_threshold arg (=10) : To set the threshold

Future Work:

• Implement the feature for save_resume.
• Work on aggregations in the online setting.

Wiki page for the same : https://github.com/VowpalWabbit/vowpal_wabbit/wiki/Privacy-Preserving-Learning