Upgrade CyberArk PAM Components (Connector Manager, Secure Tunnel, CPM & PSM) for Privilege Cloud (CPM Failover) - NETSEC

Latest

Learning, Sharing, Creating

Cybersecurity Memo

Wednesday, January 24, 2024

Upgrade CyberArk PAM Components (Connector Manager, Secure Tunnel, CPM & PSM) for Privilege Cloud (CPM Failover)

This post summzrize some notes and steps to upgrade the Privilege Cloud Connector and the components for versions 12.7 and later.

Note: Upgrading the CPM and PSM components requires downtime (typically a few minutes). We recommend performing the upgrade at a time that will have the least impact on your operations.



There are four components on our Connector Servers if you are using Privilege Cloud:
1. Secure Tunnel
2. Management Agent
3. CPM (CPM Scanner + Password Manager)
4. PSM 

Diagram


https://docs.cyberark.com/PrivCloud/Latest/en/Content/Privilege%20Cloud/PrivCloud-upgrade-connector-12.7-later.htm?tocpath=Setup%7CUpgrade%20Privilege%20Cloud%20connectors%7CUpgrade%20the%20Privilege%20Cloud%20Connector%7C_____1

 

Check .Net, CPM and PSM versions



For .Net:
  1. In the Registry Editor, open the following subkey: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\Full. If the Full subkey isn't present, then you don't have .NET Framework 4.5 or later installed.

.NET Framework versionMinimum value
.NET Framework 4.5378389
.NET Framework 4.5.1378675
.NET Framework 4.5.2379893
.NET Framework 4.6393295
.NET Framework 4.6.1394254
.NET Framework 4.6.2394802
.NET Framework 4.7460798
.NET Framework 4.7.1461308
.NET Framework 4.7.2461808
.NET Framework 4.8528040
.NET Framework 4.8.1533320



For CPM and PSM
  1. On the Connector, press Windows + R keys simultaneously to launch the Run box.

  2. In the Run box, enter appwiz.cpl, and click OK.

  3. On the Programs and Features page, select CyberArk Privilege Session Manager>CyberArk Central Policy Manager. The versions are displayed.

  4. Based on your Connector version, choose the relevant upgrade flow In this section:

For details about the version files and builds, see Release notes v14.0




Connector Management - Install and Verify


Install Connector to a new Connector server

To deploy a new connector, you first generate the installation script and then run it on the connector host machine.

To perform the following steps, your user must be assigned to the System Administrator role in Identity Administration.

  1. Sign in to the CyberArk Identity Security Platform Shared Services using the link provided in the CyberArk email.

  2. Click the service picker, and select Connector Management.

  3. On the Connectors page, click Add a connector.

  4. In the Add connector wizard > Define installation details tab define the following details for the Management Agent in the host machine:



  1. Click Next.

  2. In the Copy installation script tab, review the connector settings you defined:

Click Copy script to later copy it to the connector host machine.

The script is available for 5 minutes.

Optionally:

  • Click Renew to renew the script availability for an additional 5 minutes

  • Click Preview to view the script format

Click Close.

https://docs.cyberark.com/ConnectorManagement/Latest/en/Content/Setup/CM_AddConnector.htm?tocpath=Setup%7C_____2#Addaconnector1


Upgrade Connector Manager




Cilick upgrade:

Verify:
There is no program shows in control panel or program groups. But it does show in the services.
Only thing you can check is the exe file details to see the File Version: 1.0.606.0, in this case.




Upgrade Secure Tunnel


WARN: SecureTunnel upgrade will remove existing installtion and install new version. All existing sessions on this server will be terminated. 

https://docs.cyberark.com/ispss-deployment/latest/en/Content/Privilege%20Cloud/PrivCloud-Upgrade-SecureTunnel.htm?Highlight=upgrade%20secure%20tunnel#UpgradetheSecureTunnel1
To upgrade the Secure Tunnel for v2.0.3 and later:

CyberArk automatically saves your configuration, so there is no need for you to manually save it.

  1. Download the latest Secure Tunnel file from the CyberArk marketplace software area.



  1. Copy the Secure Tunnel installation file to the connector machine.

  2. Run the installation exe file. This upgrades the Secure Tunnel version and saves your existing configuration.

  3. You can update the Secure Tunnel configuration for any required changes, as described in Install and configure Secure Tunnel.

Verify the Secure Tunnel is functioning properly

  • In the Services list, check that the Secure Tunnel service PrivilegeCloudSecureTunnel is up and running.


Upgrade CPM and Other Components

 

At this moment, Jan 2024, it is still not able to upgrade PSM from Connector Management page. But CPM is fine now. 


Connector shows components details



Upgrade Components page

You will need to get your [email protected] credential to process. Reset the installeruser password first since it will be changed in 24 hours after reset.

This is details Failover from Active CPM to DR CPM 1. Stop the CPM services on the Active and DR CPM(s) respectively. 1. On the DR CPM, create local user PluginManagerUser if it doesn’t exist on DR CPM, copy its description from the Active CPM and get the password of the Active CPM’s PluginManagerUser using either the <tenant_name>_admin user (for standalone tenants), login name (for ISPSS tenants) or request password from CyberArk support as a last resort. Once password is acquired, use it to create the PluginManagerUser on DR CPM, or, 2. If PluginManagerUser already exists on DR CPM and you are unsure if it’s using the right password, get its password using the steps highlighted in step 2 and reset the PluginManagerUser to use the correct password. 3. Copy the vault folder from the Active CPM, this can be found typically on “[Installation Drive]\Program Files (x86)\cyberark\password manager” and paste it on the DR CPM under this file path: “[Installation Drive]\Program Files (x86)\cyberark\password manager”, if windows prompts weather the DR’s initial vault folder and its contents can be replaced with the one being copied from the Active CPM, select “yes to all”. 4. On the DR CPM run the CreateCredFile-Helper.ps1 as an administrator and reset the CPM user, use this KB as a guide on how to use the CreateCredFile-Helper.ps1: https://cyberark-customers.force.com/s/article/How-to-reset-the-CPM-and-PSM-users-in-Privilege-Cloud 5. If the CreateCredFile-Helper.ps1 prompts you to reset the apikey, select yes as well and once the CreateCredFile-Helper.ps1 runs successfully, the Active CPM would have successfully failed over to the DR CPM.



Upgrade PSM


Auto Upgrade - Directly upgrade from Connector Management
From Aug 2024, maybe a bit earlier, it is possible to upgrade PSM directly from Connetor management.




Manual Upgrade - High Level Steps

  1. Download the Privilege Cloud Connector version 14 upgrade files from the CyberArk Marketplace:
  • Privileged Session Manager-Rls-14.zip 
  • Central Policy Manager-RI14.zip 
  • Privilege Cloud Connector Unified Hardening GPO-v2.2.0.zip 
  • Privilege Cloud Connector Unified Hardening GPO-v2.2.0.txt 
 
Installed versionPatch versionDownload link
PSM 13.2 or older14.0https://www.cyberark.com/CA24-04-PSM14
CPM 13.2 or older14.0https://www.cyberark.com/CA24-04-CPM14
Privilege Cloud Connector Unified Hardening GPO2.2.0
 
  1. Follow the instructions in the documentation to upgrade to version 14.

CyberArk Documentation : Upgrade the Privilege Cloud Connector
  • https://docs.cyberark.com/PrivCloud/Latest/en/Content/Privilege%20Cloud/PrivCloud-upgrade-connector.htm

Before you upgrade the PSM component:
  • Make sure you have performed the preparatory steps described in Before you begin, in this topic.

  • Note that as part of the upgrade, legacy PSM logs are grouped in a zip file and copied to internal archive folders for future access if necessary.

To upgrade the PSM component:

  1. Open the PSM installation package you created in Prepare the Privilege CloudConnector machine:.

  2. Right-click Setup.exe, and then select Run as Administrator.

  3. The installation wizard appears. Click Next and follow these steps within the wizard:

    Tab/event

    Step

    Microsoft Visual C++ 2013 Redistributable Package (x64) errorIgnore and click Yes to Continue

    If Connector machine is domain-joined, and you logged on with a local user, the following message appears:

    • Click Yes if you are not using the RemoteApp user experience capability.

    • Click No to stop the upgrade, log on with a domain user who is a local administrator, and start the upgrade again.

    Password Vault Web Access Environment page

    Retain the default settings and click Next .

    Vault's Connection Details page

    Retain the default settings and click Next .

    Vault's Username and Password details page

    Enter the same Privilege Cloud admin credentials used for the Connector installation (<subdomain>_admin) and click Next.

    API Gateway connection details page

    Optionally, to apply the PSM automatically unlock accounts capability, enter the Privilege Cloud portal hostname in the Host field:

    <subdomain>.privilegecloud.cyberark.com

    Otherwise, click Next .

    PKI Authentication configuration page

    Optionally, to benefit from the Smart Card authentication for RDP connection capability, select Enable PKI authentication for PSM.

    Otherwise, click Next .

    If message appears, click Yes

  4. In the Hardening page, click Advanced and enter the following selections, depending on in-domain or out-of-domain hardening solution:

    Click Next .

  5. On the Update Complete page, click Finish.

     

    You can restart the Connector machine at a later stage. In any case, you must restart the Connector machine before you can use it.



Make sure reset [email protected] 's password during upgrading wizard. 


Troubleshooting


During upgrading process, here are some common errors I met:

1. ITATS053E Object PSMServer_<name> doesn't exist








02/02/2024, 12:30:02 Checking if Microsoft Visual C++ 2015-2022 x64 Redistributable Package is installed (by GUID).
02/02/2024, 12:30:02 Checking if Microsoft Visual C++ 2015-2022 x86 Redistributable Package is installed (by GUID).
02/02/2024, 12:30:02 Microsoft Visual C++ 2015-2022 Redistributable Packages are installed.
02/02/2024, 12:30:10 Checking operating system version. Additional information: 3
02/02/2024, 12:30:24 Found existing service CyberArk Privileged Session Manager
02/02/2024, 12:30:24 Start archiving logs...
02/02/2024, 12:30:26 End archive logs.
02/02/2024, 12:30:31 Installing Oracle Instant Client
02/02/2024, 12:30:31 SQLNET.ORA configuration file will be backed up to the support directory
02/02/2024, 12:30:32 A problem occurred while uninstalling deprecated version of Oracle Instant Client. Code: 1605
02/02/2024, 12:30:35 Checking the registry for X Server
02/02/2024, 12:30:36 VcXsrv Server is already installed
02/02/2024, 12:30:36 Going to Rename location : C:\Program Files (x86)\Cyberark\PSM\Hardening\PSMConfigureAppLocker.xml
02/02/2024, 12:30:36 Backing up Vault.ini
02/02/2024, 12:31:02 PSMConfigureAppLocker.xml was successfully merged with the latest CyberArk version.
02/02/2024, 12:31:02 Components folder already exists in PATH
02/02/2024, 12:31:15 The PSM remote application is already configured in your environment.
02/02/2024, 12:31:15 Loading EnvMgr
02/02/2024, 12:31:16 Vault.ini restored Successfully
02/02/2024, 12:35:08 Updating Vault environment ...
02/02/2024, 12:35:09 initializing internal process ...
02/02/2024, 12:35:09 Logging on to the Vault ...
02/02/2024, 12:35:09 Checking user permissions...
02/02/2024, 12:35:09 Checking if group PSMMaster exists.
02/02/2024, 12:35:09 PSMMaster exists. Checking if user is in the group.
02/02/2024, 12:35:09 Checking if user [email protected] is in group PSMMaster
02/02/2024, 12:35:09 User is not in group. Adding
02/02/2024, 12:35:09 Adding user [email protected] to group PSMMaster ...
02/02/2024, 12:35:09 User added to group
02/02/2024, 12:35:09 Updating Safes ...
02/02/2024, 12:35:10 Working on Safe PSM ...
02/02/2024, 12:35:11 Working on Safe PSMSessions ...
02/02/2024, 12:35:11 Working on Safe PSMLiveSessions ...
02/02/2024, 12:35:11 Working on Safe PSMUniversalConnectors ...
02/02/2024, 12:35:12 Working on Safe PSMNotifications ...
02/02/2024, 12:35:12 Storing configuration files and passwords...
02/02/2024, 12:35:13 Working on File SessionControl ...
02/02/2024, 12:35:13 Working on password PSMServer_d19777 in Safe PSM ...
02/02/2024, 12:35:13 The password object PSMServer_d19777 doesn't exist in the Safe PSM, the password will not be created in Upgrade mode.
02/02/2024, 12:35:13 Working on password PSMAdminConnect in Safe PSM ...
02/02/2024, 12:35:13 The password object PSMAdminConnect already exists in the Safe PSM, the password will not be overriden.
02/02/2024, 12:35:13 Updating Password Vault Web Access configuration files in the Vault...
02/02/2024, 12:35:22 Found PVWA version 140000
02/02/2024, 12:35:22 Start attempt to load User Management Settings from PVConfiguration.xml
02/02/2024, 12:35:22 Reading User Management settings from Password Vault Configuration...
02/02/2024, 12:35:22 Identity flag was found under PVConfiguration XML, it is being used.
02/02/2024, 12:35:22 Start attempt to load connection users object names from PVConfiguration.xml
02/02/2024, 12:35:23 Using connection user object name from PVConfiguration.xml
02/02/2024, 12:35:23 Using connection admin user object name from PVConfiguration.xml
02/02/2024, 12:35:28 Appending OIC 19c AuditFilters section
02/02/2024, 12:35:28 Not creating the General tag - tag already exists
02/02/2024, 12:35:28 Reaching to update PrivilegeCloudSessionRiskManagers group
02/02/2024, 12:35:28 About to add PSM-PTA connection component
02/02/2024, 12:35:28 PSM-PTA had been added!
02/02/2024, 12:35:28 About to add PSM-WebAppDispatcher connection component
02/02/2024, 12:35:28 PSM-WebAppSample had been added!
02/02/2024, 12:35:28 About to add PSM-MS-AzurePortal connection component
02/02/2024, 12:35:28 PSM-MS-AzurePortal had been added!
02/02/2024, 12:35:28 About to add PSM-WebAppDispatcher connection component
02/02/2024, 12:35:28 PSM-VSPHERE-New had been added!
02/02/2024, 12:35:28 About to add PSM-PVWA-v10 connection component
02/02/2024, 12:35:28 PSM-PVWA-v10 had been added!
02/02/2024, 12:35:28 About to add PSM-SQLServerMgmtStudio-Win connection component
02/02/2024, 12:35:28 PSM-SQLServerMgmtStudio-Win already exist
02/02/2024, 12:35:28 About to add PSM-SQLServerMgmtStudio-Database connection component
02/02/2024, 12:35:28 PSM-SQLServerMgmtStudio-Database already exist
02/02/2024, 12:35:28 About to add PSM-PVWA connection component
02/02/2024, 12:35:28 PSM-PVWA already exist
02/02/2024, 12:35:28 About to add PSM-PrivateArkClient connection component
02/02/2024, 12:35:28 PSM-PrivateArkClient already exist
02/02/2024, 12:36:16 Checking Secure Connect support...
02/02/2024, 12:36:16 Secure Connect feature supported.
02/02/2024, 12:36:16 Secure Connect settings found.
02/02/2024, 12:36:16 Re-logging on to the Vault ...
02/02/2024, 12:36:17 Reading category UserName on file/password PSMConnect in Safe PSM ...
02/02/2024, 12:36:17 Reading category Address on file/password PSMConnect in Safe PSM ...
02/02/2024, 12:36:17 Reading category LogonDomain on file/password PSMConnect in Safe PSM ...
02/02/2024, 12:36:17 Reading category UserName on file/password PSMAdminConnect in Safe PSM ...
02/02/2024, 12:36:18 Reading category Address on file/password PSMAdminConnect in Safe PSM ...
02/02/2024, 12:36:18 Reading category LogonDomain on file/password PSMAdminConnect in Safe PSM ...
02/02/2024, 12:36:18 Checking whether Secure Connect Safe [PSMUnmanagedSessionAccounts] exists...
02/02/2024, 12:36:18 Secure Connect Safe does not exist or Secure connect settings allready exists
02/02/2024, 12:36:18 Creating Secure Connect Safe.
02/02/2024, 12:36:18 Working on Safe PSMUnmanagedSessionAccounts ...
02/02/2024, 12:36:18 Working on Safe PSMUnmanagedSessionAccounts share agent PVWAGWAccounts ...
02/02/2024, 12:36:18 Working on Safe PSM share agent PVWAGWAccounts ...
02/02/2024, 12:36:18 Working on Safe PSMUniversalConnectors share agent PVWAGWAccounts ...
02/02/2024, 12:36:18 Updating users and groups for the Privileged Session Manager in the Vault ...
02/02/2024, 12:36:19 Working on user PSMApp_d19777 ...
02/02/2024, 12:36:19 IsCredFileInLastVersion file:C:\Program Files (x86)\Cyberark\PSM\Vault psmapp.cred TRUE.
02/02/2024, 12:36:19 Working on user PSMGw_d19777...
02/02/2024, 12:36:19 IsCredFileInLastVersion file:C:\Program Files (x86)\Cyberark\PSM\Vault psmgw.cred TRUE.
02/02/2024, 12:36:20 Creating credential file for the User PSMGw_d19777 ...
02/02/2024, 12:36:21 Working on group PSMAppUsers ...
02/02/2024, 12:36:21 Group already exists ... 
02/02/2024, 12:36:21 Working on group PSMMaster ...
02/02/2024, 12:36:21 Group already exists ... 
02/02/2024, 12:36:21 Working on group PSMLiveSessionTerminators ...
02/02/2024, 12:36:21 Group already exists ... 
02/02/2024, 12:36:21 Adding user PSMApp_d19777 to group PSMAppUsers ...
02/02/2024, 12:36:22 Adding user PSMGw_d19777 to group PVWAGWAccounts ...
02/02/2024, 12:36:23 Updating ownerships on Safes ...
02/02/2024, 12:36:24 Working on Owner PVWAAppUsers in Safe PSM ...
02/02/2024, 12:36:24 Working on Owner PVWAAppUsers in Safe PSMSessions ...
02/02/2024, 12:36:24 Working on Owner PSMApp_d19777 in Safe PVWAConfig ...
02/02/2024, 12:36:24 Working on Owner PSMAppUsers in Safe PSM ...
02/02/2024, 12:36:24 Working on Owner PSMMaster in Safe PSM ...
02/02/2024, 12:36:24 Checking if Session Admin group exists.
02/02/2024, 12:36:24 Session Admin group exists. Add all required permissions on PSM safe.
02/02/2024, 12:36:24 Working on Owner Privilege Cloud Session Admin in Safe PSM ...
02/02/2024, 12:36:25 Working on Owner PSMApp_d19777 in Safe PSMSessions ...
02/02/2024, 12:36:25 Working on Owner PSMMaster in Safe PSMSessions ...
02/02/2024, 12:36:25 Checking Owner [email protected] for Safe PSMSessions ...
02/02/2024, 12:36:25 Working on Owner PVWAAppUsers in Safe PSMLiveSessions ...
02/02/2024, 12:36:25 Working on Owner PSMAppUsers in Safe PSMLiveSessions ...
02/02/2024, 12:36:25 Working on Owner PSMMaster in Safe PSMLiveSessions ...
02/02/2024, 12:36:25 Working on Owner PSMAppUsers in Safe PSMUniversalConnectors ...
02/02/2024, 12:36:25 Working on Owner Vault Admins in Safe PSMUniversalConnectors ...
02/02/2024, 12:36:26 Working on Owner PVWAAppUsers in Safe PSMUnmanagedSessionAccounts ...
02/02/2024, 12:36:26 Working on Owner Vault Admins in Safe PSMUnmanagedSessionAccounts ...
02/02/2024, 12:36:26 Working on Owner PSMMaster in Safe PSMUnmanagedSessionAccounts ...
02/02/2024, 12:36:26 Working on Owner PSMApp_d19777 in Safe PSMUnmanagedSessionAccounts ...
02/02/2024, 12:36:26 Checking Owner [email protected] for Safe PSMUnmanagedSessionAccounts ...
02/02/2024, 12:36:26 Working on Owner PSMMaster in Safe PSMNotifications ...
02/02/2024, 12:36:26 Working on Owner PSMAppUsers in Safe PSMNotifications ...
02/02/2024, 12:36:27 Working on Owner PVWAAppUsers in Safe PSMNotifications ...
02/02/2024, 12:36:27 Updating Vault file categories ...
02/02/2024, 12:36:28 Working on file category PSMStartTime ...
02/02/2024, 12:36:28 Working on file category PSMEndTime ...
02/02/2024, 12:36:28 Working on file category PSMSourceAddress ...
02/02/2024, 12:36:28 Working on file category PSMStatus ...
02/02/2024, 12:36:28 Working on file category PSMVaultUserName ...
02/02/2024, 12:36:29 Working on file category PSMFullUserName ...
02/02/2024, 12:36:29 Working on file category PSMProtocol ...
02/02/2024, 12:36:29 Working on file category PSMClientApp ...
02/02/2024, 12:36:29 Working on file category PSMRemoteMachine ...
02/02/2024, 12:36:29 Working on file category PSMPasswordID ...
02/02/2024, 12:36:29 Working on file category PSMSafeID ...
02/02/2024, 12:36:30 Working on file category PSMRecordingType ...
02/02/2024, 12:36:30 Working on file category PSMRecordingEntity ...
02/02/2024, 12:36:30 Working on file category ProviderID ...
02/02/2024, 12:36:30 Working on file category ExpectedRecordingsList ...
02/02/2024, 12:36:30 Working on file category ActualRecordings ...
02/02/2024, 12:36:30 Working on file category RecordingUploadError ...
02/02/2024, 12:36:30 Working on file category EntityVersion ...
02/02/2024, 12:36:31 Working on file category ConnectionComponentID ...
02/02/2024, 12:36:31 Working on file category _PSMLiveSessions_1 ...
02/02/2024, 12:36:31 Working on file category _PSMLiveSessions_2 ...
02/02/2024, 12:36:31 Working on file category _PSMLiveSessions_3 ...
02/02/2024, 12:36:31 Working on file category _PSMLiveSessions_4 ...
02/02/2024, 12:36:31 Working on file category _PSMLiveSessions_5 ...
02/02/2024, 12:36:31 Working on file category DSN ...
02/02/2024, 12:36:32 Working on file category Port ...
02/02/2024, 12:36:32 Working on file category ConnectAs ...
02/02/2024, 12:36:32 Working on file category Database ...
02/02/2024, 12:36:32 Working on file category LogonDomain ...
02/02/2024, 12:36:32 Working on file category UserDN ...
02/02/2024, 12:36:32 Working on file category Location ...
02/02/2024, 12:36:33 Working on file category OwnerName ...
02/02/2024, 12:36:33 Working on file category AllowConnectToConsole ...
02/02/2024, 12:36:33 Working on file category PSMRemoteMachine ...
02/02/2024, 12:36:33 Working on file category AllowMappingLocalDrives ...
02/02/2024, 12:36:33 Working on file category PSMSingleUsePasswordObject ...
02/02/2024, 12:36:33 Working on file category TicketID ...
02/02/2024, 12:36:33 Working on file category RedirectSmartCards ...
02/02/2024, 12:36:33 Working on file category StorageLocation ...
02/02/2024, 12:36:34 Working on file category StorageObject ...
02/02/2024, 12:36:34 Removing user [email protected] from group PSMMaster.
02/02/2024, 12:36:34 Removed user from PSMMaster group.
02/02/2024, 12:36:34 Secure Connect settings creation skipped because they already exist
02/02/2024, 12:36:34 Updating PSM ini files...
02/02/2024, 12:36:35 Old PSM Version: 13.1.0.28
02/02/2024, 12:36:35 The Old 3 Parts of the PsmVersion: 131
02/02/2024, 12:36:35 Actual Recordings Folder: C:\Program Files (x86)\Cyberark\PSM\Recordings\
02/02/2024, 12:36:35 Invoking the API Key Manager...
02/02/2024, 12:36:35 apiManagerPath = C:\Program Files (x86)\Cyberark\PSM\Vault\ApiKeyManager.exe
 addupdate = add
 credfile=C:\Program Files (x86)\Cyberark\PSM\Vault\apigw.cred
 psmUser = PSMApp_d19777
 user = [email protected]
 szApiAddr = HTTPS://netsecprivilegecloud.cyberark.cloud/passwordVault/api
02/02/2024, 12:36:37 Updating Vault.ini with API GW details...
02/02/2024, 12:36:37 Updating PSM users and groups
02/02/2024, 12:36:37 Creating OS User [PSMConnect]
02/02/2024, 12:36:37 CreateOsUserHidePassword: NetUserAdd failed, code 2224, index 0
02/02/2024, 12:36:47 Creating OS User [PSMAdminConnect]
02/02/2024, 12:36:47 CreateOsUserHidePassword: NetUserAdd failed, code 2224, index 0
02/02/2024, 12:36:54 Creating OS group [PSMShadowUsers]
02/02/2024, 12:36:54 Rotating password for PSMConnect user
02/02/2024, 12:36:54 Logging on to the Vault ...
02/02/2024, 12:36:54 Adding user [email protected] to group PSMMaster ...
02/02/2024, 12:36:54 Reading category LogonDomain on file/password PSMServer_d19777 in Safe PSM ...
02/02/2024, 12:37:58 An error occurred while creating the Vault environment: ITATS053E Object PSMServer_d19777 doesn't exist.
02/02/2024, 12:37:58 Working on password PSMServer_d19777 in Safe PSM ...
02/02/2024, 12:37:58 Failed to access password object PSMServer_d19777.
02/02/2024, 12:37:58 Removing user [email protected] from group PSMMaster.
02/02/2024, 12:38:03 Retrying password rotation...
02/02/2024, 12:38:03 Logging on to the Vault ...
02/02/2024, 12:38:04 Adding user [email protected] to group PSMMaster ...
02/02/2024, 12:38:04 Reading category LogonDomain on file/password PSMServer_d19777 in Safe PSM ...
02/02/2024, 12:39:46 An error occurred while creating the Vault environment: ITATS053E Object PSMServer_d19777 doesn't exist.
02/02/2024, 12:39:46 Working on password PSMServer_d19777 in Safe PSM ...
02/02/2024, 12:39:46 Failed to access password object PSMServer_d19777.
02/02/2024, 12:39:46 Removing user [email protected] from group PSMMaster.
02/02/2024, 12:39:51 Retrying password rotation...
02/02/2024, 12:39:51 Logging on to the Vault ...
02/02/2024, 12:39:51 Adding user [email protected] to group PSMMaster ...
02/02/2024, 12:39:52 Reading category LogonDomain on file/password PSMServer_d19777 in Safe PSM ...
02/02/2024, 12:50:24 An error occurred while creating the Vault environment: ITATS053E Object PSMServer_d19777 doesn't exist.
02/02/2024, 12:50:24 Working on password PSMServer_d19777 in Safe PSM ...
02/02/2024, 12:50:25 Failed to access password object PSMServer_d19777.
02/02/2024, 12:50:25 Removing user [email protected] from group PSMMaster.
02/02/2024, 13:01:26 Password rotation failed for PSMServer_d19777, the password has not been updated. Check the logs for more details and invoke password rotation via CPM or contact CyberArk support.
02/02/2024, 13:01:26 Rotating password for PSMAdminConnect user
02/02/2024, 13:01:26 Logging on to the Vault ...
02/02/2024, 13:01:27 Adding user [email protected] to group PSMMaster ...
02/02/2024, 13:01:27 Reading category LogonDomain on file/password PSMAdminConnect in Safe PSM ...
02/02/2024, 13:01:27 Password object PSMAdminConnect references a domain users - password will not be rotated.
02/02/2024, 13:01:27 Removing user [email protected] from group PSMMaster.
02/02/2024, 13:01:28 Unloading EnvMgr
02/02/2024, 13:01:28 Registering PSM DLLs...
02/02/2024, 13:01:36 Setting folder permissions...
02/02/2024, 13:01:43 Setting RDS shadowing permissions...
02/02/2024, 13:01:44 RDS shadowing permissions was updated successfully
02/02/2024, 13:01:44 Applying security policy...
02/02/2024, 13:01:48 The hardening procedure has completed successfully
02/02/2024, 13:01:54 Service seclogon startup type was successfully updated to Automatic
02/02/2024, 13:01:59 Windows Defender exclusion for C:\Program Files (x86)\Cyberark\PSM\Components was added successfully
02/02/2024, 13:01:59 Installing service ...
02/02/2024, 13:02:00 Running PostInstallation...
02/02/2024, 13:02:00 The following steps are going to be executed:  DisableScreenSaver ConfigurePSMUsers ImproveNonRDPConnectorPerformance WebApplications
02/02/2024, 13:27:01 Failed to find '"isSucceeded":  ' in 'C:\windows\Temp\LastPSScriptLauncherOutput.log', Code: '-4'
02/02/2024, 13:27:01 Failed to find '"errorData":  ' in 'C:\windows\Temp\LastPSScriptLauncherOutput.log', Code: '-4'
02/02/2024, 13:27:01 Failed to find '"logPath":  ' in 'C:\windows\Temp\LastPSScriptLauncherOutput.log', Code: '-4'
02/02/2024, 13:27:01 Failed to find '"restartRequired":  ' in 'C:\windows\Temp\LastPSScriptLauncherOutput.log', Code: '-4'



https://www.reddit.com/r/CyberARk/comments/xyz3vt/psm_upgrade_from_120_to_126_hangs_at_post/

From powershell: 
  • dir -recurse l unblock-file


Since it is in domain, you will have to find out your connect user and admin connect user in domian.
c:\Program Files (x86)\CyberArk\PSM\Hardening
Then you will need to change hardening script PSMHardening.ps1 for following valus:


$Global:PSM_CONNECT_USER           = "COMMUNITY\svc_CArk_PSMAdmn"
$Global:PSM_ADMIN_CONNECT_USER     = "COMMUNITY\svc_CArk_PSMConnect"

Run Hardening program again

  • ./PSMHardening.ps1

If you did not change those two lines, you will get an error to say you could not find PSMInitSession.exe file. During hardening, if script asking you to remove users from remotedesktopuser group, say no. 


To fix the initial program could not start PSMInitSession.exe issue, you will need to run AppLocker Rules: (C:\Program Files (x86)\CyberArk\PSM\Hardening)

  • ./PSMConfigureAppLocker.ps1



PS C:\Program Files (x86)\Cyberark\PSM\Hardening> .\PSMConfigureAppLocker.ps1
PSM connection user is PSMConnect
PSM admin connection user is PSMAdminConnect
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\psmsshclient.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\psmprivatearkclientdispatcher.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\psmpvwadispatcher.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\mssqlmanagementstudiowindowsauthenticationdispatcher.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\psm3270client.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\psmwebformdispatcher.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\psmwinscpdispatcher.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\winscp.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\psmrealvncdispatcher.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\psmxfocus.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\psmtokenholder.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\psmsessionalert.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\psmsuspendsession.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\psmpreventwindowhide.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\psmmessagealert.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\psmwindowseventslogger.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\cyberark.psm.webappdispatcher.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\dllinjector.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\dllinjector64.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\cyberark.progressbar.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\psmticketvalidator.exe
Evaluating the dlls consumed by c:\windows\system32\conhost.exe
Evaluating the dlls consumed by c:\windows\system32\taskhostw.exe
Evaluating the dlls consumed by c:\windows\system32\wermgr.exe
Evaluating the dlls consumed by c:\program files (x86)\vcxsrv\vcxsrv.exe
Evaluating the dlls consumed by c:\program files (x86)\vcxsrv\xkbcomp.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\psmsapgui.exe
Evaluating the dlls consumed by c:\program files\google\chrome\application\chrome.exe
Evaluating the dlls consumed by c:\program files (x86)\cyberark\psm\components\chromedriver.exe
Loading new AppLocker configuration...
Configuring Application Identity service...
CyberArk AppLocker's configuration script ended successfully.
True
PS C:\Program Files (x86)\Cyberark\PSM\Hardening>


You might get following error for recording component failed to create recording file:


Grant those two domain users read and write permissions:



Notes To Upgrade CPM in Connector Manager

1. Upgrade DR one, please make sure related services disabled. 
Else it might automatically been brought up after upgrade.
Even you selected it is DR.



Failover CPM from Active Main one to DR one

You can verify version from c:\Program Files (x86) \ CyberArk \ Password Manager
  • check PMEngine.exe version





Steps for failover CPM from Main Active CPM to DR Stopped CPM:

1.  CPM service stop at Main Active CPM
  • CyberArk Central Policy Manager Scanner  - change to disabled
  • CyberArk Password Manager - change to disabled
2. Put DR Stopped / Disablled CPM services into Auto
  • CyberArk Central Policy Manager Scanner
  • CyberArk Password Manager

3. Verify following files in DR CPM servers
C:\Program Files (x86)\Cyberark\Password Manager\Vault
  • apikey.ini
  • user.ini
  • Vault.ini



apikey.ini

SecretFileType=KeyPair
SecretFileVersion=3
Username=PasswordManager
VerificationsFlag=721921
Secret=CEC73E41AF6470FFC8FD97A480D0655F22413670F1398C89B36E8BCA9B5D79811D37E55A0DDD9D6EF67CB2FF578A8AAFF396596920228549D2C043B68AA41B54E7BF94EE4B43B01B878CD892EB3EF5C870B60167057160422774B546B351BC1E16B4C7862E12A847C34B31A2C2DAE17D87FB05C26D1E11F42CCD5C896C25B55B71965465E941640195A35C787108864DCDB6722FF999D509E46698B54853CE956C64A589C40E86DDC755FB56AC5EBF5EB6E8D7EDB6F8A5C774D22E2454D67900040C2C6E48AC8EACC524AE1D9F23E642D2925848AF94BA3D58EB540ADD09E776B71211A50C6F95F85A4A20841CDCE3FE8983BB04B858CE972E990219F4DF019819B18ED4AD46BFB151C09E684D5FA1D182EEFB6F3C3A81881B85EA59649F43DBAB739AB3564DDD73E3C9EDDE094B98641815D1B3A06BAD621981500F6BAFBB0F9FECAB9FE857E110E5D1........DAB44658D3C6809E929C765CC4E5F989047A237F0FDF395C2D7560491EB6360F9EB4B1B85BDF53300
AdditionalInformation=6F9DD792F5A6C79F7607AB65A5977B150CC492D5CFEA6905B6397D388CAA2D60D6AB7DDC83F827086D40970BAC22C892010E84B0F5179735C82DFDFC2BE9127B



user.ini

SecretFileType=Password
SecretFileVersion=3
Username=PasswordManager
VerificationsFlag=721923
Secret=F63AD60C0F0C654BE06C2FFBA57E12481DA3EC8B9A8F456CA64D547AA6209EFA3873F6D8E262A1D7D8F6E079AEA9A6D088753B1B92ABDDB016536BF3F98E509F43835F74DFD4904DA63F3266A19917E52036DC8C050CBB5B1B6936042A3A9E29792FA1E91C6E1FD76A77427C99CBE8F7466330A5C7CF130967B144E7379305B409C6178F012E7C6D430351F177934F58D4CFDDAEB35AD1A054BEEDA.......................1135FAB31E98601B7B48593BA03B9CD3E02EE06
ExternalAuthentication=None
AdditionalInformation=7FF22CAA3C170DAAC93C87CA294B46E0FA41C92C93EA0B0B9C64A57D9C70E83BB5278BA3545198B3DDF64EE80BC36FBB16E0BF00FD6E0220911A5964504D7275



Vault.ini

VAULT = CPM Vault
ADDRESS=vault-51sec.privilegecloud.cyberark.cloud
PORT=1858


#-----------------------------------
# Additional parameters (optional)
#-----------------------------------
#TIMEOUT=30                        - Seconds to wait for a Vault to respond to a request
#AUTHTYPE=PA_AUTH                  - Authentication method (PA_AUTH,NT_AUTH,PKI_AUTH)
#NTAUTHAGENTNAME=    - NT Authentication Agent Name
#NTAUTHAGENTKEYFILE=    - NT Authentication Key File Name
#VAULTDN=    - Vault's Distinguished Name (PKI Authentication)
#Proxy server connection settings  - cannot be used together with BEHINDFIREWALL
#--------------------------------
#PROXYTYPE=HTTP                    - Possible values - HTTP, HTTPS, SOCKS4, SOCKS5
#PROXYADDRESS=192.333.44.55        - Proxy server IP address (mandatory when using proxy server)
#PROXYPORT=8081                    - Proxy server IP Port 
#PROXYUSER=xxx                     - User for Proxy server if NTLM authentication is required
#PROXYPASSWORD=yyy                 - Password for Proxy server if NTLM authentication is required 
#PROXYAUTHDOMAIN=NT_DOMAIN_NAME    - Domain for Proxy server if NTLM authentication is required
#BEHINDFIREWALL=NO                 - Accessing the Cyber-Ark vault via a Firewall. 
#USEONLYHTTP1=NO                   - Use only HTTP 1.0 protocol. Valid either with proxy settings Or with BEHINDFIREWALL
#NUMOFRECORDSPERSEND=15            - Number of file records that require an acknowledgement from the Vault server
#NUMOFRECORDSPERCHUNK=15           - Number of file records to transfer together in a single TCP /IP send/receive operation
#RECONNECTPERIOD=-1                - Seconds to wait before session with Vault is re-established.
#ENHANCEDSSL=NO                    - Enhanced SSL based connection (port 443) is required
#PREAUTHSECUREDSESSION=NO    - Enable pre authentication secured session 
#TRUSTSSC=NO    - Trust self-sign certificates in pre authentication secured session
#ALLOWSSCFOR3PARTYAUTH=NO    - Are self-sign certificates allowed for 3rd party authentication (like RADIUS) 
#Gateway Names
#-------------
#CIFSGATEWAY=                     - CIFS Gateway Name
#HTTPGATEWAYADDRESS=URL            - The URL of the HTTP Gateway (e.g. "https://www.cyber-ark.com/httpgw")
[API]
Addresses=https://51sec.privilegecloud.cyberark.cloud/passwordvault



4. Sync Cred


Download Cloud Tools from : https://community.cyberark.com/marketplace/s/#a352J000000GWAZQA4-a392J000002tNgLQAU
CyberArk Privilege Cloud Tools

Find folder CreateCredFile-Helper and run ps1 file to reset credentials. 
You will be asked to provide cloud admin credential which is installuser




PS C:\Users\SECSPEC-JY\Downloads\Cyberark PrivilegeCloud Tools-v15.5\CreateCredFile-Helper> .\CreateCredFile-Helper.ps1
=======================================
Starting Create CredFile helper script
Current script version 3.6
Found new version (version 3.7), Updating...
Finished Updating, relaunching the script
=======================================
Starting Create CredFile helper script
TLS 1.2 is properly configured.
Found CPM installation
Found PSM installation
================ ResetCredFile Guide ================
Displaying Only Detected CyberArk Services:
Please select an option:
1. CyberArk Password Manager (CPM)
2. CyberArk Privileged Session Manager (PSM)
Q. Press Q to Quit
Select: 1
Stopping 'CyberArk Password Manager' Service...
Service is already stopped, skipping...
Stopping 'CyberArk Central Policy Manager Scanner' Service...
Service is already stopped, skipping...
Generating CredFile: 'C:\Program Files (x86)\CyberArk\Password Manager\Vault\user.ini'
Command ended successfully
Resetting Password for User: PasswordManager
Activating User: PasswordManager
Successfully reset Password in the Vault for User: PasswordManager
Starting service 'CyberArk Password Manager'...
Successfully Started Service: CyberArk Password Manager.
Checking SystemHealth Status for User PasswordManager
CPM = PasswordManager Is : Online!
Syncing PluginManagerUser
Looking for account PluginManagerUser in vault under safe: 'PasswordManager_Accounts'
Retrieving account password
Successfully retrieved password, proceeding syncing locally.
Running test command using password from the vault with windows user 'PluginManagerUser'
Account: PluginManagerUser is out of sync!
Password for user 'PluginManagerUser' has been reset successfully.
Input JSON file created/updated at C:\Users\SECSPEC-JY\Downloads\Cyberark PrivilegeCloud Tools-v15.5\CreateCredFile-Helper\SyncCPMCompUsers_neededFrom14.2+\SyncCompUsersInput.json
Process SyncCompUsers.exe finished successfully
Starting service 'CyberArk Central Policy Manager Scanner'...
Successfully Started Service: CyberArk Central Policy Manager Scanner.
Logoff Session...
Create CredFile helper script ended
=======================================
PS C:\Users\SECSPEC-JY\Downloads\Cyberark PrivilegeCloud Tools-v15.5\CreateCredFile-Helper>


5. Check logs

Check logs file pm.log at C:\Program Files (x86)\Cyberark\Password Manager\Logs
Searching "Starting Password Manager"

28/10/2024 14:18:51 [13a8] CACPM117I Starting Password Manager 14.0.0 (14.0.0.6).
28/10/2024 14:18:52 [13a8] CACPM047E Error getting duplicated Session from Session Manager.
28/10/2024 14:20:55 [13a8] CACPM141I Password Manager initialization failed. Trying again.
28/10/2024 14:20:55 [13a8] CACPM117I Starting Password Manager 14.0.0 (14.0.0.6).
28/10/2024 14:20:55 [13a8] CACPM047E Error getting duplicated Session from Session Manager.
28/10/2024 14:22:18 [1178] CACPM100I Shutting down Password Manager.
28/10/2024 14:22:18 [13a8] CACPM141I Password Manager initialization failed. Trying again.
28/10/2024 14:22:18 [13a8] CACPM117I Starting Password Manager 14.0.0 (14.0.0.6).
28/10/2024 14:22:18 [13a8] CACPM047E Error getting duplicated Session from Session Manager.
28/10/2024 14:22:18 [13a8] CACPM118I Ending Password Manager.
29/10/2024 14:37:28 [04ac] CACPM117I Starting Password Manager 14.2.1 (14.2.1.6).
29/10/2024 14:37:29 [04ac] CACPM809I Plugin executed using 'PasswordManagerUser'.
29/10/2024 14:37:29 [04ac] CACPM812I Username: 'PluginManagerUser' exists on local machine.
29/10/2024 14:37:30 [04ac] CACPM670I Effective policy updated. ID: 32, Policy ID: 32, Platform Name: CyberArk PTA, Platform ID: CyberArkPTA, Exclusive: No, One Time: No, Expiration Period: 45, Verification Period: 7.
29/10/2024 14:37:30 [04ac] CACPM670I Effective policy updated. ID: 36, Policy ID: 36, Platform Name: Business Website, Platform ID: BusinessWebsite, Exclusive: No, One Time: No, Expiration Period: 45, Verification Period: 7.
29/10/2024 14:37:30 [04ac] CACPM670I Effective policy updated. ID: 39, Policy ID: 39, Platform Name: Generic Web App, Platform ID: GenericWebApp, Exclusive: No, One Time: No, Expiration Period: 45, Verification Period: 7.
29/10/2024 14:37:30 [04ac] CACPM670I Effective policy updated. ID: 41, Policy ID: 41, Platform Name: WIN-DOM-PSMADMIN-ACCOUNT, Platform ID: WIN-DOM-PSMADMIN-ACCOUNT, Exclusive: No, One Time: No, Expiration Period: 45, Verification Period: 7.
29/10/2024 14:37:30 [04ac] CACPM670I Effective policy updated. ID: 42, Policy ID: 42, Platform Name: EMP_WIN_DOM_RECON_ACCT, Platform ID: EMP_WIN_DOM_RECON_ACCT, Exclusive: No, One Time: No, Expiration Period: 14, Verification Period: 2.
29/10/2024 14:37:30 [04ac] CACPM670I Effective policy updated. ID: 43, Policy ID: 43, Platform Name: EMP_WIN_DOM_Private_Managed, Platform ID: EMP_WIN_DOM_Private_Managed, Exclusive: No, One Time: No, Expiration Period: 45, Verification Period: 7.
29/10/2024 14:37:30 [04ac] CACPM670I Effective policy updated. ID: 44, Policy ID: 44, Platform Name: Emp_Win_loc_desktop_admins_LCD, Platform ID: Emp_Win_loc_desktop_admins_LCD, Exclusive: No, One Time: No, Expiration Period: 45, Verification Period: 7.
29/10/2024 14:37:30 [04ac] CACPM670I Effective policy updated. ID: 45, Policy ID: 45, Platform Name: Emp_Win_loc_se


Note:

  • https://community.cyberark.com/s/article/CPM-How-to-Failover-from-Active-CPM-to-DR-CPM-and-back-Using-the-CreateCredFile-Helper-ps1




No comments:

Post a Comment