{ "content": [ { "type": "text", "text": "# sos-clean(1) (man)\n\n**Summary:** sosclean, sosmask - Obfuscate sensitive data from one or more sos reports\n\n## Flags\n\n| Flag | Long | Arg | Description |\n|------|------|-----|-------------|\n| — | --domains | DOMAINS | Provide a comma-delimited list of domain names to obfuscate, in addition to those matching the hostname of the system th |\n| — | --disable-parsers | PARSERS | Provide a comma-delimited list of parsers to disable when cleaning an archive. By de‐ fault all parsers are enabled. Not |\n| — | --skip-masking-files | FILES | Provide a comma-delimited list of files inside an archive, that cleaner should skip in cleaning. Globs like asterisk are |\n| — | --keywords | KEYWORDS | Provide a comma-delimited list of keywords to scrub in addition to the default parsers. Keywords provided by this option |\n| — | --keyword-file | FILE | Provide a file that contains a list of keywords that should be obfuscated. Each word must be specified on a newline with |\n| — | --map-file | FILE | Provide a location to a valid mapping file to use as a reference for existing obfusca‐ tion pairs. If one is found, the |\n| — | --jobs | JOBS | The number of concurrent archives to process, if more than one. If this utility is called by sos collect then the value |\n| — | --no-update | — | Do not write the mapping file contents to /etc/sos/cleaner/defaultmapping |\n| — | --keep-binary-files | — | Keep unprocessable binary files in the archive, rather than removing them. Note that binary files cannot be obfuscated, |\n| — | --archive-type | TYPE | Specify the type of archive that TARGET was generated as. When sos inspects a TARGET archive, it tries to identify what |\n\n## See Also\n\n- sos(1)\n- sos-report(1)\n- sos-collect(1)\n- sos.conf(5)\n\n## Section Outline\n\n- **NAME** (2 lines)\n- **SYNOPSIS** (1 lines) — 1 subsections\n - sos clean TARGET [options] (12 lines)\n- **DESCRIPTION** (22 lines)\n- **REQUIRED ARGUMENTS** (8 lines)\n- **OPTIONS** (1 lines) — 10 subsections\n - --domains DOMAINS (7 lines)\n - --disable-parsers PARSERS (10 lines)\n - --skip-cleaning-files, --skip-masking-files FILES (9 lines)\n - --keywords KEYWORDS (7 lines)\n - --keyword-file FILE (3 lines)\n - --map-file FILE (9 lines)\n - --jobs JOBS (6 lines)\n - --no-update (2 lines)\n - --keep-binary-files (9 lines)\n - --archive-type TYPE (25 lines)\n- **SEE ALSO** (3 lines)\n- **MAINTAINER** (2 lines) — 1 subsections\n - AUTHORS & CONTRIBUTORS (5 lines)\n\n## Full Content\n\n### NAME\n\nsosclean, sosmask - Obfuscate sensitive data from one or more sos reports\n\n### SYNOPSIS\n\n#### sos clean TARGET [options]\n\n[--domains]\n[--disable-parsers]\n[--skip-cleaning-files|--skip-masking-files]\n[--keywords]\n[--keyword-file]\n[--map-file]\n[--jobs]\n[--no-update]\n[--keep-binary-files]\n[--archive-type]\n\n### DESCRIPTION\n\nsos clean or sos mask is an sos subcommand used to obfuscate sensitive information from pre‐\nviously generated sos reports that is not covered by the standard plugin-based post process‐\ning executed during report generation, for example IP addresses.\n\nData obfuscated via this utility is done so consistently, meaning for example an IP address\nof 192.168.1.1 in an unprocessed sos report that gets obfuscated to, for example, 100.0.0.1,\nwill be changed to 100.0.0.1 in all occurrences found in the report.\n\nAdditionally, by default all such obfuscations are stored in \"maps\" that will be persistently\nsaved to /etc/sos/cleaner/defaultmapping and be re-used on subsequent runs.\n\nThis utility may also be used in-line with sos report and sos collect by specifying the\n--clean or --mask option.\n\nWhen called directly via sos clean, the obfuscated archive is written as an additional file,\nmeaning the original unprocessed report still remains on the filesystem. When called via re‐‐\nport or collect, the changes are done in-line and thus only an obfuscated archive is written\nand available. In either case, a mapping file containing the relationships between unpro‐\ncessed and obfuscated elements will be written in the same location as the resulting archive.\nThis mapping file should be kept private by system administrators.\n\n### REQUIRED ARGUMENTS\n\nTARGET\n\nThe path to the archive that is to be obfuscated. This may be an archive or an unbuilt sos\ntemporary\ndirectory. If an archive, it will first be extracted and then after obfuscation is\ncomplete re-compressed using the same compression method as the original.\n\n### OPTIONS\n\n#### --domains DOMAINS\n\nProvide a comma-delimited list of domain names to obfuscate, in addition to those\nmatching the hostname of the system that created the sos report. Subdomains that match\na domain given via this option will also be obfuscated.\n\nFor example, if --domains redhat.com is specified, then 'redhat.com' will be obfus‐\ncated, as will 'www.redhat.com' and subdomains such as 'foo.redhat.com'.\n\n#### --disable-parsers PARSERS\n\nProvide a comma-delimited list of parsers to disable when cleaning an archive. By de‐\nfault all parsers are enabled.\n\nNote that using this option is very likely to leave sensitive information in place in\nthe target archive, so only use this option when absolutely necessary or you have com‐\nplete trust in the party/parties that may handle the generated report.\n\nValid values for this option are currently: hostname, ip, ipv6, mac, keyword, and\nusername.\n\n#### --skip-cleaning-files, --skip-masking-files FILES\n\nProvide a comma-delimited list of files inside an archive, that cleaner should skip in\ncleaning.\n\nGlobs like asterisk are supported, so soscommands/host/hostname* will match all three\nusual filenames in that directory (hostname, hostnamectlstatus and hostname-f).\n\nUse this option with caution, only when being certain the given files do not contain\nany sensitive information.\n\n#### --keywords KEYWORDS\n\nProvide a comma-delimited list of keywords to scrub in addition to the default\nparsers.\n\nKeywords provided by this option will be obfuscated as \"obfuscatedwordX\" where X is an\ninteger based on the keyword's index in the parser. Note that keywords will be re‐\nplaced as both standalone words and in substring matches.\n\n#### --keyword-file FILE\n\nProvide a file that contains a list of keywords that should be obfuscated. Each word\nmust be specified on a newline within the file.\n\n#### --map-file FILE\n\nProvide a location to a valid mapping file to use as a reference for existing obfusca‐\ntion pairs. If one is found, the contents are loaded before parsing is started. This\nallows consistency between runs of this command for obfuscated pairs. By default, sos\nwill write the generated private map file to /etc/sos/cleaner/defaultmapping so that\nconsistency is maintained by default. Users may use this option to reference a map\nfile from a different run (perhaps one that was done on another system).\n\nDefault: /etc/sos/cleaner/defaultmapping\n\n#### --jobs JOBS\n\nThe number of concurrent archives to process, if more than one. If this utility is\ncalled by sos collect then the value of the jobs option for that utility will be used\nhere.\n\nDefault: 4\n\n#### --no-update\n\nDo not write the mapping file contents to /etc/sos/cleaner/defaultmapping\n\n#### --keep-binary-files\n\nKeep unprocessable binary files in the archive, rather than removing them.\n\nNote that binary files cannot be obfuscated, and thus keeping them in the archive may\nresult in otherwise sensitive information being included in the final archive. Users\nshould review any archive that keeps binary files in place before sending to a third\nparty.\n\nDefault: False (remove encountered binary files)\n\n#### --archive-type TYPE\n\nSpecify the type of archive that TARGET was generated as. When sos inspects a TARGET\narchive, it tries to identify what type of archive it is. For example, it may be a\nreport generated by sos report, or a collection of those reports generated by sos col‐‐\nlect, which require separate approaches.\n\nThis option may be useful if a given TARGET archive is known to be of a specific type,\nbut due to unknown reasons or some malformed/missing information in the archive di‐\nrectly, that is not properly identified by sos.\n\nThe following are accepted values for this option:\n\nauto Automatically detect the archive type\nreport An archive generated by sos report\ncollect An archive generated by sos collect\ninsights An archive generated by the insights-client package\n\nThe following may also be used, however note that these do not attempt to pre-load any\ninformation from the archives into the parsers. This means that, among other limita‐\ntions, items like host and domain names may not be obfuscated unless an obfuscated\nmapping already exists on the system from a previous execution.\n\ndata-dir A plain directory on the filesystem.\ntarball A generic tar archive not associated with any known tool\n\n### SEE ALSO\n\nsos(1) sos-report(1) sos-collect(1) sos.conf(5)\n\n### MAINTAINER\n\nMaintained on GitHub at https://github.com/sosreport/sos\n\n#### AUTHORS & CONTRIBUTORS\n\nSee AUTHORS file in the package documentation.\n\n\n\nThu May 21 2020 SOSCLEAN(1)\n\n" } ], "structuredContent": { "command": "sos-clean", "section": "1", "mode": "man", "summary": "sosclean, sosmask - Obfuscate sensitive data from one or more sos reports", "synopsis": "", "flags": [ { "flag": "", "long": "--domains", "arg": "DOMAINS", "description": "Provide a comma-delimited list of domain names to obfuscate, in addition to those matching the hostname of the system that created the sos report. Subdomains that match a domain given via this option will also be obfuscated. For example, if --domains redhat.com is specified, then 'redhat.com' will be obfus‐ cated, as will 'www.redhat.com' and subdomains such as 'foo.redhat.com'." }, { "flag": "", "long": "--disable-parsers", "arg": "PARSERS", "description": "Provide a comma-delimited list of parsers to disable when cleaning an archive. By de‐ fault all parsers are enabled. Note that using this option is very likely to leave sensitive information in place in the target archive, so only use this option when absolutely necessary or you have com‐ plete trust in the party/parties that may handle the generated report. Valid values for this option are currently: hostname, ip, ipv6, mac, keyword, and username." }, { "flag": "", "long": "--skip-masking-files", "arg": "FILES", "description": "Provide a comma-delimited list of files inside an archive, that cleaner should skip in cleaning. Globs like asterisk are supported, so soscommands/host/hostname* will match all three usual filenames in that directory (hostname, hostnamectlstatus and hostname-f). Use this option with caution, only when being certain the given files do not contain any sensitive information." }, { "flag": "", "long": "--keywords", "arg": "KEYWORDS", "description": "Provide a comma-delimited list of keywords to scrub in addition to the default parsers. Keywords provided by this option will be obfuscated as \"obfuscatedwordX\" where X is an integer based on the keyword's index in the parser. Note that keywords will be re‐ placed as both standalone words and in substring matches." }, { "flag": "", "long": "--keyword-file", "arg": "FILE", "description": "Provide a file that contains a list of keywords that should be obfuscated. Each word must be specified on a newline within the file." }, { "flag": "", "long": "--map-file", "arg": "FILE", "description": "Provide a location to a valid mapping file to use as a reference for existing obfusca‐ tion pairs. If one is found, the contents are loaded before parsing is started. This allows consistency between runs of this command for obfuscated pairs. By default, sos will write the generated private map file to /etc/sos/cleaner/defaultmapping so that consistency is maintained by default. Users may use this option to reference a map file from a different run (perhaps one that was done on another system). Default: /etc/sos/cleaner/defaultmapping" }, { "flag": "", "long": "--jobs", "arg": "JOBS", "description": "The number of concurrent archives to process, if more than one. If this utility is called by sos collect then the value of the jobs option for that utility will be used here. Default: 4" }, { "flag": "", "long": "--no-update", "arg": null, "description": "Do not write the mapping file contents to /etc/sos/cleaner/defaultmapping" }, { "flag": "", "long": "--keep-binary-files", "arg": null, "description": "Keep unprocessable binary files in the archive, rather than removing them. Note that binary files cannot be obfuscated, and thus keeping them in the archive may result in otherwise sensitive information being included in the final archive. Users should review any archive that keeps binary files in place before sending to a third party. Default: False (remove encountered binary files)" }, { "flag": "", "long": "--archive-type", "arg": "TYPE", "description": "Specify the type of archive that TARGET was generated as. When sos inspects a TARGET archive, it tries to identify what type of archive it is. For example, it may be a report generated by sos report, or a collection of those reports generated by sos col‐‐ lect, which require separate approaches. This option may be useful if a given TARGET archive is known to be of a specific type, but due to unknown reasons or some malformed/missing information in the archive di‐ rectly, that is not properly identified by sos. The following are accepted values for this option: auto Automatically detect the archive type report An archive generated by sos report collect An archive generated by sos collect insights An archive generated by the insights-client package The following may also be used, however note that these do not attempt to pre-load any information from the archives into the parsers. This means that, among other limita‐ tions, items like host and domain names may not be obfuscated unless an obfuscated mapping already exists on the system from a previous execution. data-dir A plain directory on the filesystem. tarball A generic tar archive not associated with any known tool" } ], "examples": [], "see_also": [ { "name": "sos", "section": "1", "url": "https://www.chedong.com/phpMan.php/man/sos/1/json" }, { "name": "sos-report", "section": "1", "url": "https://www.chedong.com/phpMan.php/man/sos-report/1/json" }, { "name": "sos-collect", "section": "1", "url": "https://www.chedong.com/phpMan.php/man/sos-collect/1/json" }, { "name": "sos.conf", "section": "5", "url": "https://www.chedong.com/phpMan.php/man/sos.conf/5/json" } ], "section_outline": [ { "name": "NAME", "lines": 2, "subsections": [] }, { "name": "SYNOPSIS", "lines": 1, "subsections": [ { "name": "sos clean TARGET [options]", "lines": 12 } ] }, { "name": "DESCRIPTION", "lines": 22, "subsections": [] }, { "name": "REQUIRED ARGUMENTS", "lines": 8, "subsections": [] }, { "name": "OPTIONS", "lines": 1, "subsections": [ { "name": "--domains DOMAINS", "lines": 7, "long": "--domains", "arg": "DOMAINS" }, { "name": "--disable-parsers PARSERS", "lines": 10, "long": "--disable-parsers", "arg": "PARSERS" }, { "name": "--skip-cleaning-files, --skip-masking-files FILES", "lines": 9, "long": "--skip-masking-files", "arg": "FILES" }, { "name": "--keywords KEYWORDS", "lines": 7, "long": "--keywords", "arg": "KEYWORDS" }, { "name": "--keyword-file FILE", "lines": 3, "long": "--keyword-file", "arg": "FILE" }, { "name": "--map-file FILE", "lines": 9, "long": "--map-file", "arg": "FILE" }, { "name": "--jobs JOBS", "lines": 6, "long": "--jobs", "arg": "JOBS" }, { "name": "--no-update", "lines": 2, "long": "--no-update" }, { "name": "--keep-binary-files", "lines": 9, "long": "--keep-binary-files" }, { "name": "--archive-type TYPE", "lines": 25, "long": "--archive-type", "arg": "TYPE" } ] }, { "name": "SEE ALSO", "lines": 3, "subsections": [] }, { "name": "MAINTAINER", "lines": 2, "subsections": [ { "name": "AUTHORS & CONTRIBUTORS", "lines": 5 } ] } ] } }