...

• CPVM acts as a server only and uses Java HTTP server

TroubleShooting

Basics

• Check logs –
  • Check MS logs (/var/log/cloudstack/management ) and
  • ssvm logs (under /var/log/cloud/) to see if you can find any exceptions in programming the certificate.
• SSVM acting as server - 
  • Via browser or any client, check whether Apache Webserver for SSVM is sending the entire chain. If not,
  • Make sure certs and key are programmed correctly on apache server by tracing command SecStorageSetupCommand. Read SSVM implementation for details.
  • Check SSVM/MS logs to see any exception.
  • Make sure certs are uploaded fine in relevant /etc/ location as discussed in implementation above.
• SSVM acting as client –
  • Ensure that realhostip.keystore (Java keystore) on destination SSVM has the root CA certificate : keytool -list -keystore realhostip.keystore -storepass vmops.com
  • Check DB has certs as you uploaded with no spaces or garbage characters, correct domain and sequence.
• Destroy SSVM - If nothing is apparent from the logs try destroying ssvm as one of the last resorts. Try this in one of the zones and see if the new ssvm which comes up solves the issues.

1. Download urls point to the old domain.
   1. Reduce the expiration duration of the urls by changing global config extract.url.expiration.interval 
   2. And change the frequency for cleanup thread through extract.url.cleanup.interval restart MS. 
   3. Wait for the cleanup thread duration and try downloading again. See whether the url is deleted.
   4. DB tables to check (don’t recommend but worst case)
      Version < 4.2 – upload table persists url. Entry is hard deleted on expiration of url.
      Version >= 4.2 –
      template_store_ref, download_url is made null on expiration of url.
      volume_store_ref, entry hard deleted on expiration of url.
2. CopyTemplate giving the exception - 
   
   Connection timed out
   Exception – ‘sun.security.validator.ValidatorException: PKIX
   certification error
   path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target’
   1. Server side validation -
   2. Via browser or any client, check whether Apache Webserver for SSVM is sending the entire chain. If not,
      1. Make sure certs and key are programmed correctly on apache server. Read ssvm implementation for details.
      2. Check ssvm/MS logs to see any exception
      3. Make sure certs are uploaded fine.
   3. Client side validation – Ensure that
      Check whether the
      realhostip.keystore (Java keystore) on destination SSVM has the root CA certificate : keytool -list -keystore realhostip.keystore -storepass vmops.com
3. Download urls are not working. 
   1. Via browser or any client, check whether Apache Webserver for SSVM is sending the entire chain

4. I don’t see the latest systemvm.iso being patched since the /usr/local/cloud/systemvm/config_ssl.sh  on ssvm is still the old one

5. I changed CPVM to work on HTTPS from HTTP, or vice-versa. It does not change.
   1. You may need to destroy and recreate your CPVM when switching between HTTP and HTTPS protocols
6. Certificate encoding - 
   1. Make sure you have correctly url encoded the certificates. To do this you can check cloud database and check the entries in keystore table and match them with the original certificates you had.
7. Uploaded wrong certificate -
   1. In case you uploaded wrong certificates for root/intermediate, you can undo that by calling the api with same name and domainsuffix. In case you uploaded wrong server certificate upload the right one through the UI keeping the same domain suffix.
8. Any other issues with ssvm -  
   
   1. Check logs - Check MS logs (/var/log/cloudstack/management ) and ssvm logs (under /var/log/cloud/) to see if you can find any exceptions in programming the certificate.
   2. Destroy ssvm - If nothing is apparent from the logs try destroying ssvm as one of the last resorts. Try this in one of the zones and see if the new ssvm which comes up solves the issues.
9. I have already uploaded certificate and chain in a prior CCP version. Am I all good?
   1. Unfortunately, no. You will need to re-upload the whole chain using the same API parameters (name, id etc.)