Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix masking commands #5071

Merged
merged 78 commits into from
Feb 26, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
318b6a6
Added check for unique indexes
Jan 12, 2019
a0ee82c
Added procedure to make up what the unique indexes are build with
Jan 12, 2019
1b03219
Added table
Jan 31, 2019
0e09376
Added extra options for data generation
Feb 1, 2019
6c9456d
Removed unused rowcount
Feb 1, 2019
24db07c
Added format for masking types
Feb 1, 2019
d609893
Added support for using formatting of values
Feb 1, 2019
ce6ef1c
Added check if any masking info is present
Feb 4, 2019
c037c87
Implemented unique index support
Feb 4, 2019
3f7adb8
Renamed variable
Feb 4, 2019
397cc1c
Revert "Renamed variable"
Feb 4, 2019
925b071
Renamed variable and applied formatiing
Feb 4, 2019
2699e2d
Fixed errors in unique value generation
Feb 4, 2019
6f1b7fb
Added variable for the unique value columns
Feb 4, 2019
14d2149
Explicitly reset the new value variable
Feb 4, 2019
e3a71f5
Moved variables to make sure they get cleared in the loop
Feb 5, 2019
6ec4f16
Removed date selection
Feb 5, 2019
08659b7
Revert "Removed date selection"
Feb 5, 2019
7bddaa2
Added section for support of max
Feb 5, 2019
04a96f0
Added part for finance
Feb 5, 2019
cbf82ad
Removed finance from other section
Feb 5, 2019
cd08733
Changed if statement to switch
Feb 5, 2019
b18ec9c
Changed switch back to if statement
Feb 5, 2019
f6f5f2e
Removed section for finance
Feb 5, 2019
cc157c2
Added checks for supported data types, masking types and sub types
Feb 5, 2019
f91b196
Moved retrieval of method and properties to begin
Feb 5, 2019
965b59d
Changed message of unsupported types
Feb 5, 2019
cf68203
Added new known types
Feb 5, 2019
24e8a98
Added extra known types
Feb 5, 2019
115e30a
Implemented support for text and ntext data types
Feb 5, 2019
ac39b64
Added ntext and text data types
Feb 5, 2019
5d2b11e
Added decimal data type
Feb 6, 2019
5daf09c
Moved data retrieval down
Feb 6, 2019
752976b
Added types and synonyms
Feb 6, 2019
1eaaf0e
Added types
Feb 6, 2019
60b066e
Added types to automatically assign the right maskingtype
Feb 6, 2019
4ca284b
Improved the check for supported data types
Feb 6, 2019
2bb85af
Fixed bug missing curly bracket
Feb 6, 2019
694cc12
Moved the retrieval of the data
Feb 6, 2019
52fba65
Changed back to SELECT * instead of selective columns
Feb 6, 2019
32a8e92
Changed the SELECT * to be selective columns from table and not object
Feb 6, 2019
a266fe8
Moved data retrieval
Feb 6, 2019
23c98b2
Removed debug query outputs
Feb 6, 2019
2934fa5
Moved where build-up
Feb 6, 2019
16ae2ef
Added special conversion to date and datetime columns
Feb 7, 2019
6a63538
Fixed bug with missing comma
Feb 7, 2019
9c8cefe
Added error record
Feb 7, 2019
df73bbb
Removed unneccesary if statement
Feb 7, 2019
25a4618
Implemented handling of temporal values generation
Feb 9, 2019
dee5393
Fixed bool handling
Feb 9, 2019
965044e
Added decimal type
Feb 9, 2019
70fd974
Added fax synonyms
Feb 9, 2019
22ded4a
Fixed bool handling
Feb 9, 2019
7e5de15
Updates test should statement
Feb 11, 2019
10b1d5e
Changed back the should statement
Feb 11, 2019
05a2873
Enabled rdp
Feb 18, 2019
578b725
Changed rdp password
Feb 18, 2019
8e58133
Formatted test
Feb 18, 2019
9103c32
Reset appveyor.yml to original settings
Feb 18, 2019
cb8b210
Changed instance
Feb 18, 2019
e76265a
Changed instance
Feb 18, 2019
a80e24d
Changed instance back
Feb 18, 2019
dbf9525
Added credential
Feb 18, 2019
c6f725f
Merge branch 'development' into fix-masking-uniqueindexes
potatoqualitee Feb 24, 2019
b096f15
0.9.773
potatoqualitee Feb 24, 2019
e6328fd
change instance
potatoqualitee Feb 24, 2019
e36b397
Make Backup-DbaDatabase use Database and ExcludeDatabase when piping …
potatoqualitee Feb 25, 2019
6f1c096
0.9.773 changelog (#5118)
alevyinroc Feb 25, 2019
8264f68
Pass Login values to Copy-DbaLogin (#5120)
wsmelton Feb 25, 2019
6b71be4
Merge branch 'fix-masking-uniqueindexes' of https://github.com/sqlcol…
Feb 25, 2019
419280a
Updated test to use other instance
Feb 25, 2019
5c4f232
Enabled RDP
Feb 25, 2019
9c6f19b
Set RDP to be started before tests
Feb 25, 2019
4a13baa
Removed RDP option again
Feb 25, 2019
7be1fbc
Added fix for datetime
Feb 25, 2019
350b378
Formatted code
Feb 25, 2019
7e3227e
Added extra check for temporal tables
Feb 25, 2019
a84c6a9
Changed test back to 2008 instance
Feb 25, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions allcommands.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -12958,6 +12958,7 @@ function Copy-DbaSsisCatalog {
)
<# Developer note: The throw calls must stay in this command #>
begin {
$ISNamespace = "Microsoft.SqlServer.Management.IntegrationServices"
function Get-RemoteIntegrationService {
param (
[Object]$Computer
Expand Down Expand Up @@ -55110,13 +55111,13 @@ function Install-DbaMaintenanceSolution {
Installs Ola Hallengren's Solution objects on RES14224 in the DBA database.
Backups will default to the default Backup Directory.
If the Maintenance Solution already exists, the script will be halted.

.EXAMPLE
PS C:\> Install-DbaMaintenanceSolution -SqlInstance RES14224 -Database DBA -BackupLocation "Z:\SQLBackup" -CleanupTime 72

This will create the Ola Hallengren's Solution objects. Existing objects are not affected in any way.


.EXAMPLE
PS C:\> $params = @{
>> SqlInstance = 'MyServer'
Expand All @@ -55129,7 +55130,7 @@ function Install-DbaMaintenanceSolution {
>> Verbose = $true
>> }
>> Install-DbaMaintenanceSolution @params

Installs Maintenance Solution to myserver in database. Adds Agent Jobs, and if any currently exist, they'll be replaced.

.EXAMPLE
Expand Down Expand Up @@ -55400,7 +55401,7 @@ function Install-DbaMaintenanceSolution {
if ($jobs) {
$jobs | ForEach-Object {
if ($Pscmdlet.ShouldProcess($instance, "Dropping job $_.name")) {
Remove-DbaAgentJob -SqlInstance $instance -Job $_.name
Remove-DbaAgentJob -SqlInstance $server -Job $_.name
}
}
}
Expand Down Expand Up @@ -85458,12 +85459,12 @@ function Set-DbaAgentSchedule {

if ($StartDate) {
Write-Message -Message "Setting job schedule start date to $StartDate for schedule $ScheduleName" -Level Verbose
$JobSchedule.StartDate = $StartDate
$JobSchedule.ActiveStartDate = $StartDate
}

if ($EndDate) {
Write-Message -Message "Setting job schedule end date to $EndDate for schedule $ScheduleName" -Level Verbose
$JobSchedule.EndDate = $EndDate
$JobSchedule.ActiveEndDate = $EndDate
}

if ($StartTime) {
Expand Down
2 changes: 1 addition & 1 deletion appveyor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ shallow_clone: true
environment:
environment: development
version: 0.9.$(appveyor_build_number)
#appveyor_rdp_password: 2odCuiKmYiem
appveyor_rdp_password: 2odCuiKmYiem
azurepasswd:
secure: ZnF3fWSDfHraMCWlHaekvWrXf3sDqY5M28HMK4236PBbNSoqP29wEhsWMQioSSYGomzgIp9vuiwR8Fc9ViNLoqq0bVcErxEojBFTaPMEzOg2ZwO9OnOTiuUEc5JkoLBv6rEBBWef/DvkFfhr1r0K0xQu6OAPYHVTCRajTZbBRNfCTUM2X2o41t+cSa7681rtnJQnB/8cAfVVnPtJ+97s8w==
azurelegacypasswd:
Expand Down
117 changes: 88 additions & 29 deletions bin/datamasking/columntypes.json
Original file line number Diff line number Diff line change
@@ -1,49 +1,54 @@
[
{
"TypeName": "Firstname",
"TypeName": "Address",
"Synonym": [
"Firstname",
"Forename"
"Address",
"Location",
"Headquarters",
"Street"
]
},
{
"TypeName": "Lastname",
"TypeName": "Bic",
"Synonym": [
"Lastname",
"Surname"
"Bic",
"BicAddress"
]
},
{
"TypeName": "Fullname",
"TypeName": "Bitcoin",
"Synonym": [
"Fullname",
"TypeName"
"Bitcoin",
"BitcoinAddress"
]
},
{
"TypeName": "Phone",
"TypeName": "Email",
"Synonym": [
"Phonenumber",
"Phone",
"Telephone",
"TelephoneNumber"
"Email",
"E-mail",
"EmailAddress"
]
},
{
"TypeName": "Address",
"TypeName": "Ethereum",
"Synonym": [
"Address",
"Location",
"Headquarters",
"Street"
"Ethereum",
"EthereumAddress"
]
},
{
"TypeName": "Zipcode",
"TypeName": "Creditcard",
"Synonym": [
"Zipcode",
"Zip",
"Postalcode"
"Creditcard",
"CreditcardNumber"
]
},
{
"TypeName": "CreditcardCVV",
"Synonym": [
"Cvv",
"CreditcardCvv"
]
},
{
Expand All @@ -53,9 +58,10 @@
]
},
{
"TypeName": "State",
"TypeName": "Company",
"Synonym": [
"State"
"Company",
"CompanyName"
]
},
{
Expand All @@ -70,6 +76,34 @@
"CountryCode"
]
},
{
"TypeName": "Firstname",
"Synonym": [
"Firstname",
"Forename"
]
},
{
"TypeName": "Fullname",
"Synonym": [
"Fullname",
"TypeName"
]
},
{
"TypeName": "Iban",
"Synonym": [
"Iban",
"IbanNumber"
]
},
{
"TypeName": "Lastname",
"Synonym": [
"Lastname",
"Surname"
]
},
{
"TypeName": "Latitude",
"Synonym": [
Expand All @@ -85,19 +119,44 @@
]
},
{
"TypeName": "Creditcard",
"TypeName": "Phone",
"Synonym": [
"Creditcard",
"CreditcardNumber"
"Fax",
"FaxNumber",
"Phonenumber",
"Phone",
"Telephone",
"TelephoneNumber"
]
},
{
"TypeName": "State",
"Synonym": [
"State"
]
},
{
"TypeName": "StateAbbr",
"Synonym": [
"StateAbbreviation",
"StateCode"
]
},
{
"TypeName": "Username",
"Synonym": [
"Login",
"LoginId",
"LoginId",
"User",
"UserId"
]
},
{
"TypeName": "Zipcode",
"Synonym": [
"Zipcode",
"Zip",
"Postalcode"
]
}
]
10 changes: 10 additions & 0 deletions changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,16 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.9.773] - 2019-02-24
### Fixed
* `Install-DbaMaintenanceSolution` now removes jobs when `-SqlCredential` is used [#5096](https://github.com/sqlcollaborative/dbatools/issues/5096)
* `Copy-DbaSsisCatalog` now properly resolves type names [#4821](https://github.com/sqlcollaborative/dbatools/issues/4821)
* Can now set schedule start & end dates with `Set-DbaAgentSchedule` [#4908](https://github.com/sqlcollaborative/dbatools/issues/4908)

## [0.9.772] - 2019-02-24
### Fixed
* `Invoke-DbaDbShrink` now properly excludes system databases when `-AllUserDatabase` is specified [#5108](https://github.com/sqlcollaborative/dbatools/issues/5108)

## [0.9.771] - 2019-02-19
### Fixed
* Azure SQL DB support for creating SQL Logins in `New-DbaLogin` [#5100](https://github.com/sqlcollaborative/dbatools/issues/5100)
Expand Down
2 changes: 1 addition & 1 deletion dbatools.psd1
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
RootModule = 'dbatools.psm1'

# Version number of this module.
ModuleVersion = '0.9.772'
ModuleVersion = '0.9.773'

# ID used to uniquely identify this module
GUID = '9d139310-ce45-41ce-8e8b-d76335aa1789'
Expand Down
19 changes: 14 additions & 5 deletions functions/Backup-DbaDatabase.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -213,18 +213,19 @@ function Backup-DbaDatabase {
Write-Message -Message 'Setting Default timestampformat' -Level Verbose
$TimeStampFormat = "yyyyMMddHHmm"
}
if ($SqlInstance.length -ne 0) {

if ($SqlInstance) {
try {
$Server = Connect-SqlInstance -SqlInstance $SqlInstance -SqlCredential $SqlCredential -AzureUnsupported
} catch {
Stop-Function -Message "Cannot connect to $SqlInstance" -ErrorRecord $_
return
}

$InputObject = $server.Databases | Where-Object Name -ne 'tempdb'

if ($Database) {
$InputObject = $server.Databases | Where-Object Name -in $Database
} else {
$InputObject = $server.Databases | Where-Object Name -ne 'tempdb'
$InputObject = $InputObject | Where-Object Name -in $Database
}

if ($ExcludeDatabase) {
Expand Down Expand Up @@ -280,13 +281,21 @@ function Backup-DbaDatabase {
}

process {
if (!$SqlInstance -and !$InputObject) {
if (-not $SqlInstance -and -not $InputObject) {
Stop-Function -Message "You must specify a server and database or pipe some databases"
return
}

Write-Message -Level Verbose -Message "$($InputObject.Count) database to backup"

if ($Database) {
$InputObject = $InputObject | Where-Object Name -in $Database
}

if ($ExcludeDatabase) {
$InputObject = $InputObject | Where-Object Name -notin $ExcludeDatabase
}

foreach ($db in $InputObject) {
$ProgressId = Get-Random
$failures = @()
Expand Down
Loading