Documentation
Rate This Document
Findability
Accuracy
Completeness
Readability
SubmitClose

Scan Rule File Usage Example

When collecting ledger and component information during system migration, you can configure a scan rule to identify new software.

Example

The default scan rule configuration file is stored in the sys-mig/resources/sysmig_disk_scan_classifier.json directory.

  1. Use the default scan rule to collect application information for system migration. For example, collect ledger information:
    1
    		 
    devkit sys-mig -c stmt -d /usr/local -o /home/report

    Command output:

    [2025-02-12 09:17:01,498] [INFO] Start to execute the command: ./devkit sys-mig -c stmt -cf /usr/local/devkit/sys-mig/resources/sysmig_default.conf -d /usr/local -o /home/report -t /usr/local/devkit/sys-mig/template/stmt_template_default.csv -l 1
[2025-02-12 09:17:01,498] [INFO] Creating file indexing...
[2025-02-12 09:17:15,893] [INFO] Parsing...
[2025-02-12 09:17:15,893] [INFO] Start scanning stmt information.
[2025-02-12 09:17:15,894] [INFO] Middleware binary start analysis.
[2025-02-12 09:17:15,894] [INFO] Start scanning middleware binary file: /usr/local/logstash-5.5.0/vendor/bundle/jruby/1.9/gems/logstash-patterns-core-4.1.1/patterns/java.
[2025-02-12 09:17:15,896] [WARNING] The file /usr/local/logstash-5.5.0/vendor/bundle/jruby/1.9/gems/logstash-patterns-core-4.1.1/patterns/java cannot be executed or no permission to execute the filepath file.
[2025-02-12 09:17:15,896] [INFO] Start scanning middleware binary file: /usr/local/nginx/sbin/nginx.
[2025-02-12 09:17:16,009] [INFO] Start scanning middleware binary file: /usr/local/src/redis-stable/src/redis-server.
[2025-02-12 09:17:16,921] [INFO] Start scanning middleware binary file: /usr/local/bin/redis-server.
[2025-02-12 09:17:19,230] [INFO] Middleware binary end of analysis.
[2025-02-12 09:17:19,230] [INFO] Middleware package start analysis.
[2025-02-12 09:17:19,230] [INFO] Start scanning middleware package file: /usr/local/tomcat/apache-tomcat-8.5.100/lib/catalina.jar.
[2025-02-12 09:17:19,296] [INFO] Start scanning middleware package file: /usr/local/hadoop-3.4.0/share/hadoop/tools/lib/kafka-clients-2.8.2.jar.
[2025-02-12 09:17:19,342] [INFO] Start scanning middleware package file: /usr/local/logstash-5.5.0/vendor/bundle/jruby/1.9/gems/logstash-input-kafka-5.1.8/vendor/jar-dependencies/runtime-jars/kafka-clients-0.10.0.1.jar.
[2025-02-12 09:17:19,353] [INFO] Start scanning middleware package file: /usr/local/logstash-5.5.0/vendor/bundle/jruby/1.9/gems/logstash-output-kafka-5.1.7/vendor/jar-dependencies/runtime-jars/kafka-clients-0.10.0.1.jar.
[2025-02-12 09:17:19,363] [INFO] Start scanning middleware package file: /usr/local/hadoop-3.4.0/share/hadoop/hdfs/lib/zookeeper-3.8.3.jar.
[2025-02-12 09:17:19,379] [INFO] Start scanning middleware package file: /usr/local/hadoop-3.4.0/share/hadoop/common/lib/zookeeper-3.8.3.jar.
[2025-02-12 09:17:19,395] [INFO] Start scanning middleware package file: /usr/local/hbase/lib/zookeeper-3.4.10.jar.
[2025-02-12 09:17:19,580] [INFO] Start scanning middleware package file: /usr/local/zookeeper-3.4.6/zookeeper-3.4.6.jar.
[2025-02-12 09:17:20,721] [INFO] Start scanning middleware package file: /usr/local/zookeeper-3.4.6/dist-maven/zookeeper-3.4.6.jar.
[2025-02-12 09:17:20,765] [INFO] Middleware package end of analysis.
Csv report is created successfully. The file is located in /home/report/sys-mig_xx.xx.xx.xx_20250212091701/stmt.csv
[2025-02-12 09:17:20,783] [INFO] Scanning information successfully completed.
  2. The scan report shows that Logstash is not identified.

  3. Go to the binary directory and configure the scan rule file.
    1
    		 
    vi sys-mig/resources/sysmig_disk_scan_classifier.json

    Add the following content in bold:

    {
  "linux_binary_classifiers": [
   {
      "cli_category": "MIDDLEWARE",
      "category": 3,
      "file_glob": "**/logstash-core/lib/logstash/version.rb",
      "version_pattern": "(?m)(?P<version>[0-9]+\\.[0-9]+\\.[0-9]+)",
      "package": "logstash",
      "supplier": "Elastic",
      "type": "BINARY",
      "metadata_type": "MIDDLEWARE_WEB_OTHER",
      "company_type": "Open-source",
      "relative_directory_hierarchy": "../../../../"
     },
    {
      "cli_category": "RUNTIME_JAVA",
      "category": 3,
      "file_glob": "**/java",
      "version_pattern": "(?m)(\\x00java\\x00|\\x00|\\x00-Djava.class.path=%s)(?P<release>[0-9]+[.0-9]*(\\x00)+)?(?P<version>\\d+(\\.\\d)+[^\\x00]+)(\\x00openjdk)?(\\x00java)?\\x00",
      "version_file": "../release",
      "package": "JDK",
      "supplier": "JDK",
      "type": "BINARY",
      "metadata_type": "JAVA_RUNTIME_BINARY",
      "company_type": "Open-source",
      "relative_directory_hierarchy": "../"
    },  
    ...
  ],
  "middleware_jar_classifiers": [
    {
      "cli_category": "MIDDLEWARE",
      "category": 3,
      "file_glob": "**/weblogic.jar",
      "version_pattern": "(?m)Implementation-Version: (?P<version>[0-9]+[.0-9]*)",
      "version_file": "MANIFEST.MF",
      "package": "WebLogic",
      "supplier": "Oracle Corporation",
      "type": "BINARY",
      "metadata_type": "MIDDLEWARE_WEB_WEBLOGIC",
      "company_type": "Commercial",
      "relative_directory_hierarchy": "../../../../../"
    },  
    ...
  ],
  "win_binary_classifiers": [
    {
      "cli_category": "RUNTIME_JAVA",
      "category": 3,
      "classify": "java_jdk_binary",
      "file_glob": "**/java.exe",
      "version_pattern": "(?m)\\x00openjdk\\x00java\\x00(?P<release>[0-9]+[.0-9]*)\\x00(?P<version>[0-9]+[^\\x00]+)\\x00",
      "package": "JDK",
      "supplier": "JDK",
      "type": "BINARY",
      "metadata_type": "JAVA_RUNTIME_BINARY_OPENJDK",
      "company_type": "Open-source",
      "relative_directory_hierarchy": "../"
    },   
    ...
  ]
}

    The software types are as follows. For details about the parameter description, see Table 1.

    • linux_binary_classifiers: software with non-compressed files, such as binary files and text files, in Linux.
    • middleware_jar_classifiers: software that needs to be decompressed in Linux. The feature files are in the JAR format.
    • win_binary_classifiers: software with non-compressed files, such as binary files and text files, in Windows.
    Table 1 Rule description

    Parameter

    Option

    Required

    Type

    Description

    cli_category

    APP_JAVA/RUNTIME_JAVA/MIDDLEWARE/DATABASE/HARDWARE/OS

    Yes

    string

    Software type, which is displayed in the command line report result.

    • APP_JAVA
    • RUNTIME_JAVA
    • MIDDLEWARE
    • DATABASE
    • HARDWARE
    • OS

    category

    1/2/3/4

    Yes

    int

    Software type, which is configured for source component collection in the WebUI.

    • 1: OS
    • 2: database
    • 3: middleware
    • 4: Java

    use_file_regex

    true/false

    No

    boolean

    Indicates whether to use regular expression matching for the feature file. The default value is false.

    • false: The file_glob file must start with **/, which is followed by the file path.
    • true: Configure a regular expression for file_glob.

    file_glob

    -

    Yes

    string

    Feature file, which is used to identify the software and match the software version.

    Example: **/nginx

    NOTE:

    The version matching priorities are file_glob and version_file in descending order.

    version_pattern

    -

    Yes

    string

    Version. Regular expression, which is used to match the feature file. If there is no regular expression, leave it blank.

    version_file

    -

    No

    string

    Version file name, which is used to match the version. This parameter is mandatory for middleware_jar_classifiers. The file contains the software version information.

    package

    -

    Yes

    string

    Official software name, for example, Nginx.

    supplier

    -

    Yes

    string

    Software vendor name, for example, Nginx, Inc.

    type

    BINARY

    Yes

    string

    Software type. Set it to BINARY.

    metadata_type

    DATABASE_OTHER/MIDDLEWARE_WEB_OTHER

    Yes

    string

    Software type.

    • Enter DATABASE_OTHER for databases.
    • Enter MIDDLEWARE_WEB_OTHER for middleware.

    company_type

    Self-owned/Open-source/Commercial/--

    Yes

    string

    Indicates whether it is open-sourced.

    relative_directory_hierarchy

    -

    Yes

    string

    Installation directory. Relative path to the feature file found based on file_glob.

    For example, if the installation directory is the parent directory of the feature file, enter ../.
  4. Collect the ledger information again.
    1
    		 
    devkit sys-mig -c stmt -d /usr/local -o /home/report

    Command output:

    [2025-02-12 09:20:56,705] [INFO] Start to execute the command: ./devkit sys-mig -c stmt -cf /usr/local/devkit/sys-mig/resources/sysmig_default.conf -d /usr/local -o /home/report -t /usr/local/devkit/sys-mig/template/stmt_template_default.csv -l 1
[2025-02-12 09:20:56,705] [INFO] Creating file indexing...
[2025-02-12 09:21:10,633] [INFO] Parsing...
[2025-02-12 09:21:10,633] [INFO] Start scanning stmt information.
[2025-02-12 09:21:10,633] [INFO] Middleware binary start analysis.
[2025-02-12 09:21:10,634] [INFO] Start scanning middleware binary file: /usr/local/logstash-5.5.0/logstash-core/lib/logstash/version.rb.
[2025-02-12 09:21:11,748] [INFO] Start scanning middleware binary file: /usr/local/logstash-5.5.0/vendor/bundle/jruby/1.9/gems/logstash-patterns-core-4.1.1/patterns/java.
[2025-02-12 09:21:11,748] [WARNING] The file /usr/local/logstash-5.5.0/vendor/bundle/jruby/1.9/gems/logstash-patterns-core-4.1.1/patterns/java cannot be executed or no permission to execute the filepath file.
[2025-02-12 09:21:11,748] [INFO] Start scanning middleware binary file: /usr/local/nginx/sbin/nginx.
[2025-02-12 09:21:11,795] [INFO] Start scanning middleware binary file: /usr/local/src/redis-stable/src/redis-server.
[2025-02-12 09:21:12,558] [INFO] Start scanning middleware binary file: /usr/local/bin/redis-server.
[2025-02-12 09:21:14,373] [INFO] Middleware binary end of analysis.
[2025-02-12 09:21:14,374] [INFO] Middleware package start analysis.
[2025-02-12 09:21:14,374] [INFO] Start scanning middleware package file: /usr/local/tomcat/apache-tomcat-8.5.100/lib/catalina.jar.
[2025-02-12 09:21:14,405] [INFO] Start scanning middleware package file: /usr/local/hadoop-3.4.0/share/hadoop/tools/lib/kafka-clients-2.8.2.jar.
[2025-02-12 09:21:14,438] [INFO] Start scanning middleware package file: /usr/local/logstash-5.5.0/vendor/bundle/jruby/1.9/gems/logstash-input-kafka-5.1.8/vendor/jar-dependencies/runtime-jars/kafka-clients-0.10.0.1.jar.
[2025-02-12 09:21:14,448] [INFO] Start scanning middleware package file: /usr/local/logstash-5.5.0/vendor/bundle/jruby/1.9/gems/logstash-output-kafka-5.1.7/vendor/jar-dependencies/runtime-jars/kafka-clients-0.10.0.1.jar.
[2025-02-12 09:21:14,456] [INFO] Start scanning middleware package file: /usr/local/hadoop-3.4.0/share/hadoop/hdfs/lib/zookeeper-3.8.3.jar.
[2025-02-12 09:21:14,471] [INFO] Start scanning middleware package file: /usr/local/hadoop-3.4.0/share/hadoop/common/lib/zookeeper-3.8.3.jar.
[2025-02-12 09:21:14,485] [INFO] Start scanning middleware package file: /usr/local/hbase/lib/zookeeper-3.4.10.jar.
[2025-02-12 09:21:14,663] [INFO] Start scanning middleware package file: /usr/local/zookeeper-3.4.6/zookeeper-3.4.6.jar.
[2025-02-12 09:21:15,765] [INFO] Start scanning middleware package file: /usr/local/zookeeper-3.4.6/dist-maven/zookeeper-3.4.6.jar.
[2025-02-12 09:21:15,804] [INFO] Middleware package end of analysis.
Csv report is created successfully. The file is located in /home/report/sys-mig_xx.xx.xx.xx_20250212092056/stmt.csv
[2025-02-12 09:21:15,820] [INFO] Scanning information successfully completed.
  5. View the scan report. Logstash is contained in the report.