diff --git a/README.md b/README.md index 16a8a54..9c6a7e4 100644 --- a/README.md +++ b/README.md @@ -4,13 +4,16 @@

-Privacy Manifest CLI tool parses an Xcode project or a Swift Package and -attempts to locate calls to Apple's required reason APIs [^1] and detect +Privacy Manifest CLI tool parses an Xcode project/workspace or a Swift Package +and attempts to locate calls to Apple's required reason APIs [^1] and detect privacy collected data frameworks [^2]. The tool detects and parses the source files of the project as well as the frameworks added in the Xcode project's Build Phase or in the Swift Package -dependencies. It does not perform any sort of analysis beyond the simple +dependencies. It also detects any frameworks / static libraries and checks if +they are included in the third-party SDK list that Apple has provided [^3]. + +The tool does not perform any sort of analysis beyond the simple line-by-line check for the existence of the method calls or symbols that Apple has already published. @@ -32,7 +35,7 @@ After installing the tool to the `/usr/local/bin` directory, you can invoke it from any directory using the following command: ``` -privacy-manifest analyze --project path/to/project --reveal-occurrences +privacy-manifest analyze --project path/to/project --reveal-occurrences --output path ``` The `path/to/project` can be a relative or an absolute path to the `.xcodeproj` @@ -43,6 +46,10 @@ regarding the occurrences of the required reason APIs / privacy collected data frameworks in your codebase, highlighting the file and the line where a call has been detected. +The `--output` flag is optional and if specified, a `PrivacyInfo.xcprivacy` +property list file will be generated to that directory based on the detected +required reason APIs and from the responses of the user. + ## Example Below is the console output from the [VLC iOS OSS](https://github.com/videolan/vlc-ios). @@ -51,10 +58,7 @@ Below is the console output from the [VLC iOS OSS](https://github.com/videolan/v ## Future implementations -There are several ideas that can be explored here, beyond the typical performance -optimizations: The tool can output the report to HTML, or attempt to generate -an initial privacy manifest based on the user's input (maybe it can be more -interactive). +The tool can output the occurrences report to HTML for better readability. On top of that, the list of third-party crash frameworks can be updated so that it can inform the user when such framework is detected (there is a related TODO @@ -69,8 +73,8 @@ an unused piece of code. Furthermore, there might also be cases where something has not been included in the parsing process. This tool gives you a high-level overview of the different required reason APIs -and privacy collected data frameworks your project / package uses, so always do -your own research after using this tool, to confirm the findings. +and privacy collected data frameworks your project, workspace or package uses, +so always do your own research after using this tool, to confirm the findings. ## License @@ -78,3 +82,4 @@ Licensed under Apache License 2.0, see [LICENSE](LICENSE) file. [^1]: https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_use_of_required_reason_api [^2]: https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_data_use_in_privacy_manifests). +[^3]: https://developer.apple.com/support/third-party-SDK-requirements/ diff --git a/Sources/PrivacyManifest/Constants.swift b/Sources/PrivacyManifest/Constants.swift index 0f58627..29c2cc5 100644 --- a/Sources/PrivacyManifest/Constants.swift +++ b/Sources/PrivacyManifest/Constants.swift @@ -26,6 +26,7 @@ struct CliSyntaxColor { let PACKAGE_SWIFT_FILENAME = "Package.swift" let XCODE_PROJECT_PATH_EXTENSION = "xcodeproj" +let XCODE_WORKSPACE_PATH_EXTENSION = "xcworkspace" enum RequiredReasonKey: CaseIterable { case FILE_TIMESTAMP_APIS_KEY @@ -64,6 +65,226 @@ enum RequiredReasonKey: CaseIterable { } } + var privacyManifestKey: String? { + switch self { + case .FILE_TIMESTAMP_APIS_KEY: + return "NSPrivacyAccessedAPICategoryFileTimestamp" + case .SYSTEM_BOOT_APIS_KEY: + return "NSPrivacyAccessedAPICategorySystemBootTime" + case .DISK_SPACE_APIS_KEY: + return "NSPrivacyAccessedAPICategoryDiskSpace" + case .ACTIVE_KEYBOARD_APIS_KEY: + return "NSPrivacyAccessedAPICategoryActiveKeyboards" + case .USER_DEFAULTS_APIS_KEY: + return "NSPrivacyAccessedAPICategoryUserDefaults" + default: + return nil + } + } + + var reasons: [String: String] { + switch self { + case .DISK_SPACE_APIS_KEY: + return [ + "85F4.1" : """ +Declare this reason to display disk space information to the +person using the device. +Disk space may be displayed in units of information (such as bytes) +or units of time combined with a media type (such as minutes of HD video). + +Information accessed for this reason, or any derived information, +may not be sent off-device. There is an exception that allows the app to +send disk space information over the local network to another device +operated by the same person only for the purpose of displaying disk space +information on that device; this exception only applies if the user has +provided explicit permission to send disk space information, +and the information may not be sent over the Internet. +""", + "E174.1" : """ +Declare this reason to check whether there is sufficient disk space to +write files, or to check whether the disk space is low so that the app can +delete files when the disk space is low. The app must behave differently +based on disk space in a way that is observable to users. + +Information accessed for this reason, or any derived information, +may not be sent off-device. There is an exception that allows the app to +avoid downloading files from a server when disk space is insufficient. +""", + "7D9E.1" : """ +Declare this reason to include disk space information in an optional bug +report that the person using the device chooses to submit. +The disk space information must be prominently displayed to the person as +part of the report. + +Information accessed for this reason, or any derived information, +may be sent off-device only after the user affirmatively chooses to submit +the specific bug report including disk space information, +and only for the purpose of investigating or responding to the bug report. +""", + "B728.1" : """ +Declare this reason if your app is a health research app, and you access +this API category to detect and inform research participants about low disk +space impacting the research data collection. + +Your app must comply with App Store Review Guideline §5.1.3. +Your app must not offer any functionality other than providing information +about and allowing people to participate in health research. +""" + ] + case .FILE_TIMESTAMP_APIS_KEY: + return [ + "DDA9.1" : """ +Declare this reason to display file timestamps to the person using the device. + +Information accessed for this reason, or any derived information, +may not be sent off-device. +""", + "C617.1" : """ +Declare this reason to access the timestamps, size, or other metadata of files +inside the app container, app group container, or the app’s CloudKit container. +""", + "3B52.1" : """ +Declare this reason to access the timestamps, size, or other metadata of files +or directories that the user specifically granted access to, such as using +a document picker view controller. +""", + "0A2A.1" : """ +Declare this reason if your third-party SDK is providing a wrapper function +around file timestamp API(s) for the app to use, and you only access the +file timestamp APIs when the app calls your wrapper function. This reason may +only be declared by third-party SDKs. This reason may not be declared if your +third-party SDK was created primarily to wrap required reason API(s). + +Information accessed for this reason, or any derived information, +may not be used for your third-party SDK’s own purposes or sent off-device +by your third-party SDK. +""" + ] + case .SYSTEM_BOOT_APIS_KEY: + return [ + "35F9.1" : """ +Declare this reason to access the system boot time in order to measure the +amount of time that has elapsed between events that occurred within the app +or to perform calculations to enable timers. + +Information accessed for this reason, or any derived information, +may not be sent off-device. There is an exception for information about the +amount of time that has elapsed between events that occurred within the app, +which may be sent off-device. +""", + "8FFB.1" : """ +Declare this reason to access the system boot time to calculate absolute +timestamps for events that occurred within your app, such as events related +to the UIKit or AVFAudio frameworks. + +Absolute timestamps for events that occurred within your app may be sent +off-device. System boot time accessed for this reason, or any other information +derived from system boot time, may not be sent off-device. +""", + "3D61.1" : """ +Declare this reason to include system boot time information in an optional +bug report that the person using the device chooses to submit. +The system boot time information must be prominently displayed to the person +as part of the report. + +Information accessed for this reason, or any derived information, +may be sent off-device only after the user affirmatively chooses to submit +the specific bug report including system boot time information, +and only for the purpose of investigating or responding to the bug report. +""" + ] + case .ACTIVE_KEYBOARD_APIS_KEY: + return [ + "3EC4.1" : """ +Declare this reason if your app is a custom keyboard app, and you access +this API category to determine the keyboards that are active on the device. + +Providing a systemwide custom keyboard to the user must be the primary +functionality of the app. + +Information accessed for this reason, or any derived information, +may not be sent off-device. +""", + "54BD.1" : """ +Declare this reason to access active keyboard information to present +the correct customized user interface to the person using the device. +The app must have text fields for entering or editing text and must behave +differently based on active keyboards in a way that is observable to users. + +Information accessed for this reason, or any derived information, +may not be sent off-device. +""" + ] + case .USER_DEFAULTS_APIS_KEY: + return [ + "CA92.1" : """ +Declare this reason to access user defaults to read and write information +that is only accessible to the app itself. + +This reason does not permit reading information that was written by +other apps or the system, or writing information that can be accessed by +other apps. +""", + "1C8F.1" : """ +Declare this reason to access user defaults to read and write information +that is only accessible to the apps, app extensions, and App Clips that +are members of the same App Group as the app itself. + +This reason does not permit reading information that was written by apps, +app extensions, or App Clips outside the same App Group or by the system. +Your app is not responsible if the system provides information from the +global domain because a key is not present in your requested domain while +your app is attempting to read information that apps, app extensions, +or App Clips in your app’s App Group write. + +This reason also does not permit writing information that can be accessed +by apps, app extensions, or App Clips outside the same App Group. +""", + "C56D.1" : """ +Declare this reason if your third-party SDK is providing a wrapper function +around user defaults API(s) for the app to use, and you only access the +user defaults APIs when the app calls your wrapper function. +This reason may only be declared by third-party SDKs. +This reason may not be declared if your third-party SDK was created primarily +to wrap required reason API(s). + +Information accessed for this reason, or any derived information, +may not be used for your third-party SDK’s own purposes or sent off-device +by your third-party SDK. +""", + "AC6B.1" : """ +Declare this reason to access user defaults to read the +com.apple.configuration.managed key to retrieve the managed app configuration +set by MDM, or to set the com.apple.feedback.managed key to store +feedback information to be queried over MDM, as described in the +Apple Mobile Device Management Protocol Reference documentation. +""" + ] + case .CORELOCATION_FRAMEWORK_KEY: + return [:] // TODO + case .HEALTHKIT_FRAMEWORK_KEY: + return [:] // TODO + case .CRASH_FRAMEWORK_KEY: + return [:] // TODO + case .CONTACTS_FRAMEWORK_KEY: + return [:] // TODO + case .THIRD_PARTY_SDK_KEY: + return [ + "reason" : """ + ++---------------------------------------------------------------------------------+ +| You must include the privacy manifest for any of the above SDKs when you submit | +| new apps in App Store Connect that include those SDKs, or when you submit an | +| app update that adds one of the listed SDKs as part of the update. | +| Signatures are also required in these cases where the listed SDKs are used | +| as binary dependencies. Any version of a listed SDK, as well as any SDKs that | +| repackage those on the list, are included in the requirement. | ++---------------------------------------------------------------------------------+ + +"""] + } + } + var link: String { switch self { case .FILE_TIMESTAMP_APIS_KEY: @@ -101,7 +322,6 @@ let ALLOWED_EXTENSIONS = [ // Look through the code for the listed strings (Case Sensitive) let APIS_TO_CHECK: [String: [RequiredReasonKey]] = [ // Ref: https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_use_of_required_reason_api - "creationDate": [.FILE_TIMESTAMP_APIS_KEY], "modificationDate": [.FILE_TIMESTAMP_APIS_KEY], "fileModificationDate": [.FILE_TIMESTAMP_APIS_KEY], @@ -130,7 +350,6 @@ let APIS_TO_CHECK: [String: [RequiredReasonKey]] = [ "UserDefaults": [.USER_DEFAULTS_APIS_KEY], // Ref: https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_data_use_in_privacy_manifests - "import CoreLocation": [.CORELOCATION_FRAMEWORK_KEY], "#import ": [.CORELOCATION_FRAMEWORK_KEY], @@ -158,7 +377,6 @@ let APIS_TO_CHECK: [String: [RequiredReasonKey]] = [ // listed strings (Case Insensitive) let SDKS_TO_CHECK: [String: RequiredReasonKey] = [ // Ref: https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_data_use_in_privacy_manifests - "sentry-cocoa": .CRASH_FRAMEWORK_KEY, "Sentry": .CRASH_FRAMEWORK_KEY, "Instabug": .CRASH_FRAMEWORK_KEY, @@ -175,7 +393,7 @@ let SDKS_TO_CHECK: [String: RequiredReasonKey] = [ "Abseil": .THIRD_PARTY_SDK_KEY, "AFNetworking": .THIRD_PARTY_SDK_KEY, "Alamofire": .THIRD_PARTY_SDK_KEY, - "AppAuth": .THIRD_PARTY_SDK_KEY, + "AppAuth": .THIRD_PARTY_SDK_KEY, // also covers: GTMAppAuth "BoringSSL": .THIRD_PARTY_SDK_KEY, "openssl_grpc": .THIRD_PARTY_SDK_KEY, "Capacitor": .THIRD_PARTY_SDK_KEY, @@ -187,27 +405,20 @@ let SDKS_TO_CHECK: [String: RequiredReasonKey] = [ "DKPhotoGallery": .THIRD_PARTY_SDK_KEY, "FBAEMKit": .THIRD_PARTY_SDK_KEY, "FBLPromises": .THIRD_PARTY_SDK_KEY, - "FBSDKCoreKit": .THIRD_PARTY_SDK_KEY, - "FBSDKCoreKit_Basics": .THIRD_PARTY_SDK_KEY, + "FBSDKCoreKit": .THIRD_PARTY_SDK_KEY, // also covers: FBSDKCoreKit_Basics "FBSDKLoginKit": .THIRD_PARTY_SDK_KEY, "FBSDKShareKit": .THIRD_PARTY_SDK_KEY, "file_picker": .THIRD_PARTY_SDK_KEY, "FirebaseABTesting": .THIRD_PARTY_SDK_KEY, "FirebaseAuth": .THIRD_PARTY_SDK_KEY, - "FirebaseCore": .THIRD_PARTY_SDK_KEY, - "FirebaseCoreDiagnostics": .THIRD_PARTY_SDK_KEY, - "FirebaseCoreExtension": .THIRD_PARTY_SDK_KEY, - "FirebaseCoreInternal": .THIRD_PARTY_SDK_KEY, + "FirebaseCore": .THIRD_PARTY_SDK_KEY, // also covers: FirebaseCoreDiagnostics, FirebaseCoreExtension, FirebaseCoreInternal "FirebaseCrashlytics": .THIRD_PARTY_SDK_KEY, "FirebaseDynamicLinks": .THIRD_PARTY_SDK_KEY, "FirebaseFirestore": .THIRD_PARTY_SDK_KEY, "FirebaseInstallations": .THIRD_PARTY_SDK_KEY, "FirebaseMessaging": .THIRD_PARTY_SDK_KEY, "FirebaseRemoteConfig": .THIRD_PARTY_SDK_KEY, - "Flutter": .THIRD_PARTY_SDK_KEY, - "flutter_inappwebview": .THIRD_PARTY_SDK_KEY, - "flutter_local_notifications": .THIRD_PARTY_SDK_KEY, - "fluttertoast": .THIRD_PARTY_SDK_KEY, + "Flutter": .THIRD_PARTY_SDK_KEY, // also covers: flutter_inappwebview, flutter_local_notifications, fluttertoast "FMDB": .THIRD_PARTY_SDK_KEY, "geolocator_apple": .THIRD_PARTY_SDK_KEY, "GoogleDataTransport": .THIRD_PARTY_SDK_KEY, @@ -215,27 +426,20 @@ let SDKS_TO_CHECK: [String: RequiredReasonKey] = [ "GoogleToolboxForMac": .THIRD_PARTY_SDK_KEY, "GoogleUtilities": .THIRD_PARTY_SDK_KEY, "grpcpp": .THIRD_PARTY_SDK_KEY, - "GTMAppAuth": .THIRD_PARTY_SDK_KEY, "GTMSessionFetcher": .THIRD_PARTY_SDK_KEY, "hermes": .THIRD_PARTY_SDK_KEY, "image_picker_ios": .THIRD_PARTY_SDK_KEY, - "IQKeyboardManager": .THIRD_PARTY_SDK_KEY, - "IQKeyboardManagerSwift": .THIRD_PARTY_SDK_KEY, + "IQKeyboardManager": .THIRD_PARTY_SDK_KEY, // also covers: IQKeyboardManagerSwift "Kingfisher": .THIRD_PARTY_SDK_KEY, "leveldb": .THIRD_PARTY_SDK_KEY, "Lottie": .THIRD_PARTY_SDK_KEY, "MBProgressHUD": .THIRD_PARTY_SDK_KEY, "nanopb": .THIRD_PARTY_SDK_KEY, - "OneSignal": .THIRD_PARTY_SDK_KEY, - "OneSignalCore": .THIRD_PARTY_SDK_KEY, - "OneSignalExtension": .THIRD_PARTY_SDK_KEY, - "OneSignalOutcomes": .THIRD_PARTY_SDK_KEY, + "OneSignal": .THIRD_PARTY_SDK_KEY, // also covers: OneSignalCore, OneSignalExtension, OneSignalOutcomes "OpenSSL": .THIRD_PARTY_SDK_KEY, "OrderedSet": .THIRD_PARTY_SDK_KEY, - "package_info": .THIRD_PARTY_SDK_KEY, - "package_info_plus": .THIRD_PARTY_SDK_KEY, - "path_provider": .THIRD_PARTY_SDK_KEY, - "path_provider_ios": .THIRD_PARTY_SDK_KEY, + "package_info": .THIRD_PARTY_SDK_KEY, // also covers: package_info_plus + "path_provider": .THIRD_PARTY_SDK_KEY, // also covers: path_provider_ios "Promises": .THIRD_PARTY_SDK_KEY, "Protobuf": .THIRD_PARTY_SDK_KEY, "Reachability": .THIRD_PARTY_SDK_KEY, @@ -254,8 +458,7 @@ let SDKS_TO_CHECK: [String: RequiredReasonKey] = [ "SwiftyJSON": .THIRD_PARTY_SDK_KEY, "Toast": .THIRD_PARTY_SDK_KEY, "UnityFramework": .THIRD_PARTY_SDK_KEY, - "url_launcher": .THIRD_PARTY_SDK_KEY, - "url_launcher_ios": .THIRD_PARTY_SDK_KEY, + "url_launcher": .THIRD_PARTY_SDK_KEY, // also covers: url_launcher_ios "video_player_avfoundation": .THIRD_PARTY_SDK_KEY, "wakelock": .THIRD_PARTY_SDK_KEY, "webview_flutter_wkwebview": .THIRD_PARTY_SDK_KEY diff --git a/Sources/PrivacyManifest/ProjectParser.swift b/Sources/PrivacyManifest/ProjectParser.swift index 322b917..01f2792 100644 --- a/Sources/PrivacyManifest/ProjectParser.swift +++ b/Sources/PrivacyManifest/ProjectParser.swift @@ -45,6 +45,9 @@ class ProjectParser { final func parseFiles(filePathsForParsing: [Path], targetName: String, spinner: Spinner) throws { + guard filePathsForParsing.count > 0 else { + return + } let targetGroup = DispatchGroup() var parsed = 1 for filePath in filePathsForParsing { @@ -94,7 +97,7 @@ class ProjectParser { self.requiredAPIsLock.unlock() } - final func process(revealOccurrences: Bool) { + final func process(revealOccurrences: Bool) -> [RequiredReasonKey: Set] { print("---") requiredAPIs.sorted(by: { if $0.value.count == $1.value.count { @@ -105,9 +108,6 @@ class ProjectParser { } }).forEach { (key, list) in print("\(CliSyntaxColor.WHITE_BOLD)\(key.description) (\(list.count) \(list.count == 1 ? "occurrence" : "occurrences")\(CliSyntaxColor.END))") - if list.count > 0 { - print("\(CliSyntaxColor.CYAN)⚓︎ \(key.link)\(CliSyntaxColor.END)") - } if !revealOccurrences { return @@ -143,6 +143,8 @@ class ProjectParser { print("\n") } + + return requiredAPIs } } diff --git a/Sources/PrivacyManifest/SpinnerStreams.swift b/Sources/PrivacyManifest/SpinnerStreams.swift index 713e4ba..405b7ea 100644 --- a/Sources/PrivacyManifest/SpinnerStreams.swift +++ b/Sources/PrivacyManifest/SpinnerStreams.swift @@ -9,56 +9,84 @@ import Foundation import Spinner -// Display several different spinner outputs concurrently +// Displays several different spinner outputs at once class ConcurrentSpinnerStream { // The array of concurrent silent spinner streams to manage - var silentSpinners: [SilentSpinnerStream] = [] + var silentSpinnerStreams: [SilentSpinnerStream] = [] + + private static let MAX_OUTPUT_LINES = 10 private var previousRows = 0 - private let queue = DispatchQueue(label: "concurrent.spinner.stream") + + // Serial queue that ensures that console output is serial. + private let queue = DispatchQueue(label: "stream.queue") private let group = DispatchGroup() + // Serial queue that ensures that there are no race conditions on spinner + // calls. + private let spinnersQueue = DispatchQueue(label: "spinner.queue") + private let spinnersGroup = DispatchGroup() + init() { - // Hides the cursor from console - print("\u{001B}[?25l", terminator: "") - fflush(stdout) + hideCursor() } // Renders the added silent spinner streams // NOTE: Only call it from within the serial qeueue private func render() { - guard silentSpinners.count > 0 else { + guard silentSpinnerStreams.count > 0 else { return } + // Move cursor at the beginning of the previously rendered string if previousRows > 0 { print("\u{001B}[\(previousRows)F", terminator: "") } // Clear from cursor to end of screen print("\u{001B}[0J", terminator: "") + // Generate the buffer var buffer = "" - silentSpinners.forEach { silentSpinner in + var linesRendered = 0 + + silentSpinnerStreams.sorted().forEach { silentSpinner in + if linesRendered > Self.MAX_OUTPUT_LINES { + return + } buffer.append(silentSpinner.buffer + "\n") + linesRendered += 1 } + print("\(buffer)", terminator: "") fflush(stdout) - previousRows = silentSpinners.count + + previousRows = linesRendered } + func hideCursor() { + // Hides the cursor from console + print("\u{001B}[?25l", terminator: "") + fflush(stdout) + } + func waitAndShowCursor() { // Wait until all async requests have been printed + _ = spinnersGroup.wait(timeout: .distantFuture) _ = group.wait(timeout: .distantFuture) + // Shows the cursor to console print("\u{001B}[?25h", terminator: "") fflush(stdout) + + silentSpinnerStreams.removeAll() + previousRows = 0 } // Adds a silent spinner stream func add(stream: SilentSpinnerStream) { queue.async(group: group, execute: DispatchWorkItem(block: { - self.silentSpinners.append(stream) + self.silentSpinnerStreams.append(stream) })) } @@ -67,39 +95,41 @@ class ConcurrentSpinnerStream { /// /// - Parameters: /// - work: The task to be completed asynchronously within the serial queue - /// - render: Whether after the asynchronous execution of the task, the added silent spinner - /// streams should be rendered or not. - fileprivate func executeAsync(work: @escaping () -> Void, - render: Bool = false) { + private func executeSpinnerAsync(work: @escaping () -> Void) { + spinnersQueue.async(group: spinnersGroup, + execute: DispatchWorkItem(block: { + work() + })) + } + + fileprivate func executeSpinnerStreamAsync(work: @escaping () -> Void) { queue.async(group: group, execute: DispatchWorkItem(block: { work() - if render { - self.render() - } + self.render() })) } func start(spinner: Spinner) { - executeAsync { + executeSpinnerAsync { spinner.start() } } func success(spinner: Spinner, _ message: String) { - executeAsync { + executeSpinnerAsync { spinner.success(message) } } func message(spinner: Spinner, _ message: String) { - executeAsync { + executeSpinnerAsync { spinner.message(message) } } func error(spinner: Spinner, _ message: String) { - executeAsync { + executeSpinnerAsync { spinner.error(message) } } @@ -112,12 +142,22 @@ class ConcurrentSpinnerStream { } // Writes the spinner stream to a buffer, instead of the stdout -class SilentSpinnerStream: SpinnerStream { +class SilentSpinnerStream: SpinnerStream, Comparable { + static func == (lhs: SilentSpinnerStream, rhs: SilentSpinnerStream) -> Bool { + lhs.lastUpdated == rhs.lastUpdated + } + + static func < (lhs: SilentSpinnerStream, rhs: SilentSpinnerStream) -> Bool { + lhs.lastUpdated > rhs.lastUpdated + } + var buffer = "" + var lastUpdated: TimeInterval private var concurrentStream: ConcurrentSpinnerStream init(concurrentStream: ConcurrentSpinnerStream) { + self.lastUpdated = Date().timeIntervalSince1970 self.concurrentStream = concurrentStream concurrentStream.add(stream: self) } @@ -126,14 +166,15 @@ class SilentSpinnerStream: SpinnerStream { guard string.count > 0 else { return } - concurrentStream.executeAsync(work: { + concurrentStream.executeSpinnerStreamAsync(work: { + self.lastUpdated = Date().timeIntervalSince1970 // If the message contains a success or an error character, treat // it as the final message for that spinner stream. guard !self.buffer.contains("✔") && !self.buffer.contains("✖") else { return } self.buffer = string - }, render: true) + }) } func hideCursor() { } diff --git a/Sources/PrivacyManifest/XcodeProjectParser.swift b/Sources/PrivacyManifest/XcodeProjectParser.swift index c5e8be9..650fa87 100644 --- a/Sources/PrivacyManifest/XcodeProjectParser.swift +++ b/Sources/PrivacyManifest/XcodeProjectParser.swift @@ -12,14 +12,37 @@ import PathKit import XcodeProj +// Parses all Xcode projects contained in a Xcode workspace +class XcodeWorkspaceParser: XcodeProjectParser { + override func parse() throws { + print("---") + + let xcworkspace = try XCWorkspace(path: projectPath) + + try xcworkspace.data.children.forEach { element in + print("\(CliSyntaxColor.WHITE_BOLD)\(element.location.path)\(CliSyntaxColor.END)") + + concurrentStream.hideCursor() + + let projectPath = projectPath.parent() + Path(element.location.path) + + try parseProject(projectPath) + } + } +} + // Parses all targets' supported source files and frameworks. class XcodeProjectParser: ProjectParser { override func parse() throws { - let xcodeproj = try XcodeProj(path: projectPath) - print("---") - xcodeproj.pbxproj.nativeTargets.forEach { target in + try parseProject(projectPath) + } + + fileprivate func parseProject(_ path: Path) throws { + let xcodeproj = try XcodeProj(path: path) + + try xcodeproj.pbxproj.nativeTargets.forEach { target in guard let productType = target.productType else { return } @@ -29,16 +52,46 @@ class XcodeProjectParser: ProjectParser { return } + if productType == .staticLibrary + || productType == .staticFramework + || productType == .framework + || productType == .xcFramework { + let spinner = concurrentStream.createSilentSpinner(with: "Looking up \(CliSyntaxColor.GREEN)\(target.name)'s\(CliSyntaxColor.END) name...") + concurrentStream.start(spinner: spinner) + queue.async(group: group, + execute: DispatchWorkItem(block: { + SDKS_TO_CHECK.forEach { (key, value) in + let markedResults = Self.mark(searchString: key, + in: target.name, + lineNumber: nil, + caseInsensitive: true, + requiredReasonKeys: [value]) + guard let firstResult = markedResults.first?.1 else { + return + } + let highlightedCode = "\(Self.addBracketsToString(firstResult.line,around: firstResult.range))" + let foundInBuildPhase = "Found \(highlightedCode)." + self.updateRequiredAPIs(value, + with: PresentedResult(filePath: foundInBuildPhase)) + } + self.concurrentStream.success(spinner: spinner, + "Looked up \(CliSyntaxColor.GREEN)\(target.name)'s\(CliSyntaxColor.END) name") + })) + } + // https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_data_use_in_privacy_manifests target.buildPhases.forEach { phase in guard phase.buildPhase == .frameworks else { return } + guard let files = phase.files, files.count > 0 else { + return + } let spinner = concurrentStream.createSilentSpinner(with: "Parsing \(CliSyntaxColor.GREEN)\(target.name)'s\(CliSyntaxColor.END) Frameworks Build Phase...") concurrentStream.start(spinner: spinner) queue.async(group: group, execute: DispatchWorkItem(block: { - phase.files?.forEach({ file in + files.forEach({ file in guard let fullFileName = file.file?.name else { return } @@ -62,13 +115,14 @@ class XcodeProjectParser: ProjectParser { })) } + let sourceFiles = try target.sourceFiles() let spinner = concurrentStream.createSilentSpinner(with: "Parsing \(CliSyntaxColor.GREEN)\(target.name)'s\(CliSyntaxColor.END) source files...") concurrentStream.start(spinner: spinner) queue.async(group: group, execute: DispatchWorkItem(block: { do { var filePathsForParsing: [Path] = [] - try target.sourceFiles().forEach { file in + try sourceFiles.forEach { file in guard let path = file.path, let ext = Path(path).extension, ALLOWED_EXTENSIONS.contains(ext) @@ -95,5 +149,7 @@ class XcodeProjectParser: ProjectParser { _ = group.wait(timeout: .distantFuture) concurrentStream.waitAndShowCursor() + + print("---") } } diff --git a/Sources/PrivacyManifest/main.swift b/Sources/PrivacyManifest/main.swift index 1506259..6b722bc 100644 --- a/Sources/PrivacyManifest/main.swift +++ b/Sources/PrivacyManifest/main.swift @@ -16,19 +16,35 @@ struct PrivacyManifest: ParsableCommand { commandName: "privacy-manifest", abstract: "Privacy Manifest tool", discussion: """ -An easy and fast way to parse your whole Xcode project or Swift Package in -order to find whether your codebase makes use of Apple's required reason APIs +An easy and fast way to parse your whole Xcode project, Xcode workspace or +Swift Package in order to find whether your codebase makes use of Apple's +required reason APIs (https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_use_of_required_reason_api) or privacy collected data (https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_data_use_in_privacy_manifests). !!! Disclaimer: This tool must *not* be used as the only way to generate the privacy manifest. Do your own research !!! """, - version: "0.0.10", + version: "0.0.11", subcommands: [Analyze.self]) } struct Analyze: ParsableCommand { + private static let PRIVACYINFO_FILENAME = "PrivacyInfo.xcprivacy" + + // The data structure of the generated PrivacyInfo.xcprivacy file + struct PrivacyManifestDataStructure: Encodable { + struct PrivacyAccessedAPIType: Encodable { + var nsPrivacyAccessedAPIType: String + var nSPrivacyAccessedAPITypeReasons: [String] + } + var nsPrivacyTracking: Bool + var nsPrivacyTrackingDomains: [String] + var nsPrivacyCollectedDataTypes: [[String:String]] + var nsPrivacyAccessedAPITypes: [PrivacyAccessedAPIType] + } + enum DetectedProjectType { case xcodeProject(Path) + case xcodeWorkspace(Path) case swiftPackage(Path) case directory(Path) } @@ -37,23 +53,29 @@ struct Analyze: ParsableCommand { commandName: "analyze", abstract: "Analyzes the project to detect privacy aware API usage", discussion: """ -Supports Xcode projects (.xcodeproj) and Swift Packages (Package.swift). - -On the Xcode projects, the tool also parses the Framework's Build Phase of each -target to detect the Libraries used. +Supports Xcode projects (.xcodeproj), Xcode workspaces (.xcworkspace) and +Swift Packages (Package.swift). !!! Disclaimer: This tool must *not* be used as the only way to generate the privacy manifest. Do your own research !!! """ ) @Option(name: .long, help: """ -Either the (relative/absolute) path to the project's .xcodeproj (e.g. path/to/MyProject.xcodeproj) or to the Package.swift (e.g. path/to/Package.swift). +Either the (relative/absolute) path to the project's +.xcodeproj(e.g. path/to/MyProject.xcodeproj), +.xcworkspace (e.g. path/to/MyWorkspace.xcworkspace) or +Package.swift (e.g. path/to/Package.swift). """) private var project : String @Flag(name: .long, help: "Reveals the API occurrences on each file.") var revealOccurrences: Bool = false + @Option(name: .long, help: """ +The path to the directory where the privacy manifest file will be generated (Optional). +""") + var output: String? + func run() throws { let projectPath = Path(project).absolute() var detectedProjectType: DetectedProjectType? @@ -64,8 +86,14 @@ Either the (relative/absolute) path to the project's .xcodeproj (e.g. path/to/My else if projectPath.extension == XCODE_PROJECT_PATH_EXTENSION { detectedProjectType = .xcodeProject(projectPath) } + else if projectPath.extension == XCODE_WORKSPACE_PATH_EXTENSION { + detectedProjectType = .xcodeWorkspace(projectPath) + } else if projectPath.isDirectory { - let children = try projectPath.children() + // Reverse sort the children paths so that xcworkspace is parsed + // first if both .xcodeproj and .xcworkspace are found in the same + // path. + let children = try projectPath.children().sorted().reversed() guard children.count > 0 else { print("\(CliSyntaxColor.RED)Empty directory: \(projectPath)\(CliSyntaxColor.END)") return @@ -74,7 +102,10 @@ Either the (relative/absolute) path to the project's .xcodeproj (e.g. path/to/My if detectedProjectType != nil { return } - if childPath.extension == XCODE_PROJECT_PATH_EXTENSION { + else if childPath.extension == XCODE_WORKSPACE_PATH_EXTENSION { + detectedProjectType = .xcodeWorkspace(childPath) + } + else if childPath.extension == XCODE_PROJECT_PATH_EXTENSION { detectedProjectType = .xcodeProject(childPath) } else if childPath.lastComponent == PACKAGE_SWIFT_FILENAME { @@ -91,31 +122,51 @@ Either the (relative/absolute) path to the project's .xcodeproj (e.g. path/to/My return } + var requiredAPIs: [RequiredReasonKey: Set]? + switch detectedProjectType { - case .swiftPackage(let path): - print("Swift Package detected: \(CliSyntaxColor.WHITE_BOLD)\(path)\(CliSyntaxColor.END)") - try measure { - let swiftPackage = SwiftPackageProjectParser(with: path) - try swiftPackage.parse() - swiftPackage.process(revealOccurrences: revealOccurrences) - } - break - case .xcodeProject(let path): - print("Xcode project detected: \(CliSyntaxColor.WHITE_BOLD)\(path)\(CliSyntaxColor.END)") - try measure { - let xcodeProject = XcodeProjectParser(with: path) - try xcodeProject.parse() - xcodeProject.process(revealOccurrences: revealOccurrences) - } - break - case .directory(let path): - print("Directory detected: \(CliSyntaxColor.WHITE_BOLD)\(path)\(CliSyntaxColor.END)") - try measure { - let xcodeProject = DirectoryProjectParser(with: path) - try xcodeProject.parse() - xcodeProject.process(revealOccurrences: revealOccurrences) - } - break + case .swiftPackage(let path): + print("Swift Package detected: \(CliSyntaxColor.WHITE_BOLD)\(path)\(CliSyntaxColor.END)") + let swiftPackage = SwiftPackageProjectParser(with: path) + try measure { + try swiftPackage.parse() + } + requiredAPIs = swiftPackage.process(revealOccurrences: revealOccurrences) + case .xcodeProject(let path): + print("Xcode project detected: \(CliSyntaxColor.WHITE_BOLD)\(path)\(CliSyntaxColor.END)") + let xcodeProject = XcodeProjectParser(with: path) + try measure { + try xcodeProject.parse() + } + requiredAPIs = xcodeProject.process(revealOccurrences: revealOccurrences) + case .xcodeWorkspace(let path): + print("Xcode workspace detected: \(CliSyntaxColor.WHITE_BOLD)\(path)\(CliSyntaxColor.END)") + let xcodeWorkspace = XcodeWorkspaceParser(with: path) + try measure { + try xcodeWorkspace.parse() + } + requiredAPIs = xcodeWorkspace.process(revealOccurrences: revealOccurrences) + case .directory(let path): + print("Directory detected: \(CliSyntaxColor.WHITE_BOLD)\(path)\(CliSyntaxColor.END)") + let xcodeProject = DirectoryProjectParser(with: path) + try measure { + try xcodeProject.parse() + } + requiredAPIs = xcodeProject.process(revealOccurrences: revealOccurrences) + } + + if let output = output, + let requiredAPIs = requiredAPIs { + print("---") + + let outputPath = Path(output) + + guard outputPath.isDirectory else { + print("\(CliSyntaxColor.RED)Error: Output path not a directory\(CliSyntaxColor.END)") + return + } + generateManifest(requiredAPIs, + outputPath: Path(output) + Self.PRIVACYINFO_FILENAME) } } @@ -124,6 +175,94 @@ Either the (relative/absolute) path to the project's .xcodeproj (e.g. path/to/My let result = try clock.measure(function) print("Execution took \(result)") } + + func generateManifest(_ requiredAPIs: [RequiredReasonKey: Set], + outputPath: Path) { + var manifestReasons: [PrivacyManifestDataStructure.PrivacyAccessedAPIType] = [] + + requiredAPIs.forEach { (key, value) in + guard value.count > 0, key.reasons.count > 0 else { + return + } + + if key == .THIRD_PARTY_SDK_KEY, let reason = key.reasons.first { + var results: Set = Set() + value.forEach { result in + results.update(with: result.filePath) + } + print("\n\(CliSyntaxColor.WHITE_BOLD)WARNING:\(CliSyntaxColor.END) The following third-party SDKs were detected:\n") + print("* \(results.joined(separator: "\n* "))") + print("\(CliSyntaxColor.WHITE_BOLD)\(reason.value)\(CliSyntaxColor.END)") + print("\(CliSyntaxColor.CYAN)⚓︎ \(key.link)\(CliSyntaxColor.END)\n") + print("Hit ENTER to continue: ", terminator: "") + _ = readLine() + return + } + + guard let privacyManifestKey = key.privacyManifestKey else { + return + } + + print("\n\(CliSyntaxColor.WHITE_BOLD)\(value.count) \(value.count == 1 ? "occurrence" : "occurrences") for \(key.description)\(CliSyntaxColor.END). Available reasons:\n") + + var index = 0 + let reasonKeys = [String](key.reasons.keys) + reasonKeys.forEach { reasonKey in + guard let value = key.reasons[reasonKey] else { + return + } + print(""" +\(CliSyntaxColor.WHITE_BOLD)\(index+1).\(CliSyntaxColor.END) \(value)\n +""") + index += 1 + } + + print("\(CliSyntaxColor.CYAN)⚓︎ \(key.link)\(CliSyntaxColor.END)\n") + + print("Enter the values that match your case (comma separated, enter for none): ", + terminator: "") + + var manifestReasonKeys: [String] = [] + + if let input = readLine() { + let values = input.components(separatedBy: ",") + values.forEach { value in + guard let index = Int(value.trimmingCharacters(in: .whitespaces)), + index - 1 >= 0 && index - 1 < reasonKeys.count else { + return + } + let reasonKey = reasonKeys[index - 1] + manifestReasonKeys.append(reasonKey) + } + } + + if manifestReasonKeys.count > 0 { + manifestReasons.append(PrivacyManifestDataStructure.PrivacyAccessedAPIType( + nsPrivacyAccessedAPIType: privacyManifestKey, + nSPrivacyAccessedAPITypeReasons: manifestReasonKeys)) + } + } + + print("\n") + + guard manifestReasons.count > 0 else { + print("\(CliSyntaxColor.YELLOW)No reasons were provided, Privacy Manifest file generation was skipped.\(CliSyntaxColor.END)") + return + } + + let privacyManifestDataStructure = PrivacyManifestDataStructure( + nsPrivacyTracking: false, + nsPrivacyTrackingDomains: [], + nsPrivacyCollectedDataTypes: [], + nsPrivacyAccessedAPITypes: manifestReasons) + + do { + try PropertyListEncoder().encode(privacyManifestDataStructure).write(to: outputPath.url) + print("\(CliSyntaxColor.GREEN)✔\(CliSyntaxColor.END) Privacy Manifest file was generated successfully at \(CliSyntaxColor.WHITE_BOLD)\(outputPath.absolute())\(CliSyntaxColor.END)") + } catch { + print("\(CliSyntaxColor.RED)✖ Error generating Privacy Manifest file: \(error)\(CliSyntaxColor.END)") + } + } } PrivacyManifest.main()