Skip to main content

Community

Other: Consistency of JSON Layout during Channel Definition Exports

Planned

Please sign in to leave a comment.

Comments

9 comments

  • Mark Van de Wiel User

    Hi Blessy,

    It is a bit of a surprise to me that you experience a different layout of the JSON between different definition exports. I would expect if you consider the same items for export you'd get the same layout.

    Have you considered using a utility like (Linux) jq to achieve a consistent format? E.g. simply run:

    jq . <export file>

    Use the output of that command as the basis for comparisons.

    Please let me know if that works for you.

    Thank you,
    Mark.

    Hi Mark,

    Sorry if my message was not clear. I exported the same channel twice (in an interval of 1hour without changing anything with the channel), and the order of the items were different. 

    Note that this has no impact on the following import. However, it is causing us a lot of false positives with our CICD as the files keeps being identified as changes even though nothing really changed. 

    Our HVR implementation is on Windows, so we do not have access to try your suggestion. 

    Let me know if you want us to provide an example of these.

    Thanks,

    Blessy

  • Mark Van de Wiel User

    Thank you Blessy for the extra information.

    I have asked the engineering team to take a look at this.

    Mark.

  • Mark Van de Wiel User

    Blessy,

    Can you please share a couple of files that demonstrate the issue you face?

    Thank you,
    Mark.

    Hi Mark, Please find the code before and after. Thanks, Blessy

    First Version 
    {
      "export_header": {
        "creator": "jdai0001",
        "hvr_version": "6.2.0/0",
        "api_version": "latest",
        "description": "Exported with UI",
        "tstamp": "2025-04-04T09:16:38.004Z",
        "original_hub": "hvr_sit_hub",
        "type": "definition_changes"
      },
      "changes": [
        {
          "add_channel": {
            "description": "Ultracs DWLNAPP File Conversion - Step 1",
            "loc_groups": {
              "SOURCE": {
                "members": [
                  "fs_ultracs"
                ]
              },
              "TARGET": {
                "members": [
                  "fs_hvr_conv"
                ]
              }
            },
            "tables": {},
            "actions": [
              {
                "loc_scope": "*",
                "table_scope": "*",
                "type": "AgentPlugin",
                "params": {
                  "Command": "D:/appl/hvr6/hvr_home/plugin/agent/del_to_parquet.py",
                  "ExecOnHub": 1
                }
              },
              {
                "loc_scope": "SOURCE",
                "table_scope": "*",
                "type": "Capture",
                "params": {
                  "Pattern": "DWLNAPP.txt"
                }
              },
              {
                "loc_scope": "TARGET",
                "table_scope": "*",
                "type": "ColumnProperties",
                "params": {
                  "Extra": 1,
                  "Name": "hvr_is_deleted",
                  "SoftDelete": 1
                }
              },
              {
                "loc_scope": "TARGET",
                "table_scope": "*",
                "type": "ColumnProperties",
                "params": {
                  "Datatype": "timestamp with time zone",
                  "Extra": 1,
                  "IntegrateExpression": "{hvr_integ_tstamp}",
                  "Name": "hvr_change_time",
                  "Precision": "3"
                }
              },
              {
                "loc_scope": "TARGET",
                "table_scope": "*",
                "type": "Integrate",
                "params": {
                  "RenameExpression": "{hvr_cap_filename}"
                }
              },
              {
                "loc_scope": "TARGET",
                "table_scope": "*",
                "type": "Restrict",
                "params": {
                  "CompareCondition": "{hvr_is_deleted} = 0",
                  "Context": "!compare_deleted"
                }
              },
              {
                "loc_scope": "TARGET",
                "table_scope": "*",
                "type": "Restrict",
                "params": {
                  "Context": "refresh_keep_deletes",
                  "RefreshCondition": "{hvr_is_deleted} = 0"
                }
              }
            ],
            "channel": "ult_s3_aa1"
          }
        }
      ]
    }

     

     

    Second version
    {
      "export_header": {
        "creator": "emaria01",
        "hvr_version": "6.2.0/0",
        "api_version": "latest",
        "description": "Exported with UI",
        "tstamp": "2025-05-12T23:41:00.793Z",
        "original_hub": "hvr_sit_hub",
        "type": "definition_changes"
      },
      "changes": [
        {
          "add_channel": {
            "description": "Ultracs DWLNAPP File Conversion - Step 1",
            "loc_groups": {
              "SOURCE": {
                "members": [
                  "fs_ultracs"
                ]
              },
              "TARGET": {
                "members": [
                  "fs_hvr_conv"
                ]
              }
            },
            "tables": {},
            "actions": [
              {
                "loc_scope": "*",
                "table_scope": "*",
                "type": "AgentPlugin",
                "params": {
                  "Command": "D:/appl/hvr6/hvr_home/plugin/agent/src_to_parquet.py",
                  "ExecOnHub": 1
                }
              },
              {
                "loc_scope": "SOURCE",
                "table_scope": "*",
                "type": "Capture",
                "params": {
                  "AccessDelay": "20",
                  "Pattern": "DWLNAPP.txt"
                }
              },
              {
                "loc_scope": "TARGET",
                "table_scope": "*",
                "type": "ColumnProperties",
                "params": {
                  "Datatype": "timestamp with time zone",
                  "Extra": 1,
                  "IntegrateExpression": "{hvr_integ_tstamp}",
                  "Name": "hvr_change_time",
                  "Precision": "3"
                }
              },
              {
                "loc_scope": "TARGET",
                "table_scope": "*",
                "type": "ColumnProperties",
                "params": {
                  "Extra": 1,
                  "Name": "hvr_is_deleted",
                  "SoftDelete": 1
                }
              },
              {
                "loc_scope": "TARGET",
                "table_scope": "*",
                "type": "Integrate",
                "params": {
                  "RenameExpression": "{hvr_cap_filename}"
                }
              },
              {
                "loc_scope": "TARGET",
                "table_scope": "*",
                "type": "Restrict",
                "params": {
                  "Context": "refresh_keep_deletes",
                  "RefreshCondition": "{hvr_is_deleted} = 0"
                }
              },
              {
                "loc_scope": "TARGET",
                "table_scope": "*",
                "type": "Restrict",
                "params": {
                  "CompareCondition": "{hvr_is_deleted} = 0",
                  "Context": "!compare_deleted"
                }
              }
            ],
            "channel": "ult_s3_aa1"
          }
        }
      ]
    }

     

    Hi Mark,
    Any update on this.

    Thanks,

    Blessy

  • Mark Van de Wiel User

    Blessy,

    Apologies for the radio silence. We transitioned our ticketing system and alerts were not propagated to the new system. Hence i missed your updates.

    Let me discuss with engineering this week and respond again.

    Thank you.
    Mark.

  • Mark Van de Wiel User

    Blessy,

    Sadly a resolution to this will be delayed by a few weeks. It is summer vacation on the northern hemisphere with multiple engineers out on vacation.

    Thank you for your patience.
    Mark.

  • Mark Van de Wiel User

    Hi Blessy,

    FYI engineering are now working on this request.

    Thank you for your patience.
    Mark.