budget_management_en.md

Budget Management Design

Background

In order to meet the needs of users to manage the privacy budget of datasets, configure SQL queries' privacy budget, it is necessary to develop the budget management service.

Goals

Finish the development of the following functions:

• Management of table budget information (including set and get)
• Privacy budget recovery: timed recovery tasks following privacy budget recovery policy set by users
• Supports users to set an allocated privacy budget for individual query results per query
• Record the privacy budget cost per query and store it to MySQL

Overall design

Budget management service structure

Interface design

Preconditions:

1. The database and table that store budget info have been created before DPSQL services starts.

Table 1: budget storage table

column_names	description	type	remarks
prefix	host information of the databse	string	Cannot be None
db_name	database name	string	cannot be None
table_name	table name	string	cannot be None
total_budget	privacy budget	float	cannot be None
consumed_budget	total consumed privacy budget	float	cannot be None
recover_cycle	Privacy budget recovery policy, that is, the number of interval days between two adjacent recoveries	int	cannot be None
exhausted_strategy	Policies when privacy budget is exhausted	"reject" or "allow",   #Reject means to reject the query; allow means to allow the query, but this query's budget cost is not recorded	Cannot be None
create_time	The time when this record is created	string	cannot be None
last_update_time	The time when this record was last updated	string	cannot be None
last_recover_time	The time when  the privacy budget of this record was last recovered	string	Cannot be None
slack	The set of parameters for calculating the total  privacy budget cost	float	Cannot be None; Users cannot directly update these parameters with http interfaces
num_dpcall		int
sum_eps		float
sum_del		float
sum_sq_eps		float
sum_exp_eps		float
prod_del		float

2. register blueprint app.py

app.register_blueprint(budget, url_prefix='/api/v1/budget')

###Set budget information

• Request path:/set
• Request method: post
• request parameters

column_name	description	type	remarks
prefix	host information of the databse	string	Cannot be None
db_name	database name	string	cannot be None
table_name	table name	string	cannot be None
total_budget	privacy budget	float	optimal, default = 1000.0
recover_cycle	Privacy Budget Recovery Strategy	int, the number of days to restore the privacy budget from the time the service was running	optimal, default = 30
exhausted_strategy	Policies when privacy budgets run out	string, "reject" or "allow",   #Reject means to reject the query; allow means to allow the query, but this query consumption is not recorded	optimal, default=“reject”

@views.route('/set', methods=['POST'])
def set_budget_info():
    
    return response

# Response
// succeed
{
    status:{
        "code": 200, 
        "Message": "succeed"
    }
}

// error
{
  status:{
        "code": 1,
        "Message": error_info 
    }
}

Get budget information

• Request path:/get
• Request method: get
• request parameters

column_name	description	type	remarks
prefix	host information of the databse	string	Cannot be None
db_name	database name	string	cannot be None
table_name	table name	string	cannot be None

@views.route('/get', methods=['GET'])
def get_budget_info(request):
    
    return response

# Response
// succeed
{
     status:{
            "code": 200,
            "Message": "succeed",
            "data": {
                "prefix"：string，
                "db_name"：string，
                "table_name": string,
                "total_budget": float,
                "residual_budget": float,
                ...
            }
     }
}

// error
{
  status:{
        "code": 1,
        "Message": "budget info does not exist"
    }
}

Users set budget for each answer of the query

Users set dpconfig in sql's key to achieve this goal.

key = {
    "sql": sql,
    "dbconfig": {
        "reader": "AnalysisBase",
        "psm": "olap.clickhouse.player_test01_lfxlq.service.lf",
        "database": "rangers",
        "sha256sum": "Sha256 checksum of database dp config",
        "queryOption": {
            "skip_cache": 1,
            "with_column_type": True
        }
    },
    "queryconfig": {
        "traceid": "traceid",
    },
    "dpconfig": { 
        "dp_method": "Gauss", # "Laplace" or "Gauss". Default value is "Laplace".
        "budget_setting": {
            "epsilon":  float，  # optimal, default = 0.9
            "delt": float,  # optimal, defaut=1e-8
        },
    },
    "extra": {
        "debug": True,
    }
}