EJBCA support, development and maintenance by

OCSP Installation

This section contains installation instructions for running EJBCA as an external OCSP responder where a separate key pair and certificate is used to sign OCSP responses on behalf of a CA. By default, each CA can sign OCSP responses out of the box using the CA key pair. See 'OCSP Architecture' to determine if you also need an external OCSP responder.

If you are not using the responder with EJBCA you can skip the section about building and configuring EJBCA.

bin/ejbca.sh ca importcacert ManagementCA /home/user/adminca1.pem
bin/ejbca.sh roles addadmin "Super Administrator Role" ManagementCA WITH_COMMONNAME TYPE_EQUALCASE SuperAdmin

- Key Usage: Digital Signature
- Extended Key Usage: OCSPSigner

bin/ejbca.sh cryptotoken

bin/ejbca.sh keybind

bin/ejbca.sh ocsp setdefaultresponder <DN>

bin/ejbca.sh ca importcacert OCSP_CA /home/tomas/tmp/OCSP_CA.pem
bin/ejbca.sh cryptotoken create OCSPCryptoToken foo123 true SoftCryptoToken true
bin/ejbca.sh cryptotoken generatekey OCSPCryptoToken ocspsignkey RSA2048
bin/ejbca.sh keybind create OCSP_CA_KeyBinding OcspKeyBinding DISABLED null OCSPCryptoToken ocspsignkey SHA256WithRSA -nonexistingisgood=false -includecertchain=true
bin/ejbca.sh keybind gencsr OCSP_CA_KeyBinding csr.pem

(send the csr to OCSP_CA and get signed certificate back)

bin/ejbca.sh keybind import OCSP_CA_KeyBinding ocsp.pem 
bin/ejbca.sh keybind setstatus OCSP_CA_KeyBinding ACTIVE

(now your OCSP key binding is active and can be used to sign OCSP queries)

openssl ocsp -issuer OCSP_CA.pem -CAfile OCSP_CA.pem -cert ocsp.pem -req_text -url http://localhost:8080/ejbca/publicweb/status/ocsp
OCSP Request Data:
    Version: 1 (0x0)
    Requestor List:
        Certificate ID:
          Hash Algorithm: sha1
          Issuer Name Hash: E4A38A2DB963CAA8EEDFE4FBD396EE1E9B82FC19
          Issuer Key Hash: EF9C1460AEEF978FCFD30A3E7B1A2CE0BF36F9AB
          Serial Number: 054C3D7EA9E92EA5
    Request Extensions:
        OCSP Nonce: 
            0410ED1DFBA35756BBBF033FABB4055166E0
Response verify OK
ocsp.pem: good
	This Update: Feb  5 12:58:43 2014 GMT

openssl ocsp -issuer OCSP_CA.pem -CAfile OCSP_CA.pem -cert qwe.pem -req_text -url http://localhost:8080/ejbca/publicweb/status/ocsp
OCSP Request Data:
    Version: 1 (0x0)
    Requestor List:
        Certificate ID:
          Hash Algorithm: sha1
          Issuer Name Hash: E4A38A2DB963CAA8EEDFE4FBD396EE1E9B82FC19
          Issuer Key Hash: EF9C1460AEEF978FCFD30A3E7B1A2CE0BF36F9AB
          Serial Number: 5033A405556C4C26
    Request Extensions:
        OCSP Nonce: 
            0410CF2AA349C1EAF562650CE38FC9AD75B7
Response verify OK
qwe.pem: unknown
	This Update: Feb  5 13:00:51 2014 GMT

java -jar dist-crlFetch/crlFetch-0.9.jar

openssl ocsp -issuer OCSP_CA.pem -CAfile OCSP_CA.pem -cert qwe.pem -req_text -url http://localhost:8080/ejbca/publicweb/status/ocsp
OCSP Request Data:
    Version: 1 (0x0)
    Requestor List:
        Certificate ID:
          Hash Algorithm: sha1
          Issuer Name Hash: E4A38A2DB963CAA8EEDFE4FBD396EE1E9B82FC19
          Issuer Key Hash: EF9C1460AEEF978FCFD30A3E7B1A2CE0BF36F9AB
          Serial Number: 5033A405556C4C26
    Request Extensions:
        OCSP Nonce: 
            0410F767B9AE83322DBC8DE85783348C118F
Response verify OK
qwe.pem: good
	This Update: Feb  5 12:59:59 2014 GMT

java -jar dist-revoke/revoke-0.9.jar "CN=OCSP_CA" < revoke.txt

openssl ocsp -issuer OCSP_CA.pem -CAfile OCSP_CA.pem -cert qwe.pem -req_text -url http://localhost:8080/ejbca/publicweb/status/ocsp
OCSP Request Data:
    Version: 1 (0x0)
    Requestor List:
        Certificate ID:
          Hash Algorithm: sha1
          Issuer Name Hash: E4A38A2DB963CAA8EEDFE4FBD396EE1E9B82FC19
          Issuer Key Hash: EF9C1460AEEF978FCFD30A3E7B1A2CE0BF36F9AB
          Serial Number: 5033A405556C4C26
    Request Extensions:
        OCSP Nonce: 
            041096D62CECE66549C45D798E88B4D03593
Response verify OK
qwe.pem: revoked
	This Update: Feb  5 13:10:10 2014 GMT
	Reason: certificateHold
	Revocation Time: Feb  5 13:09:48 2014 GMT

Automated renewal of an OCSP signer via CA's WebService (a.k.a. Re-keying)

Re-keying allows the OCSP responder to generate new signing keys and obtain a new certificate for these keys from the CA's WebService. Re-keying is configured in the ocsp.properties configuration file. The client authentication SSL certificate is configured as an AuthenticationKeyBinding in the AdminGUI (or using the EJB CLI).

Re-keying can be either automatic or manual. Automatic re-keying allows you to specify the maximum expiration period in seconds before the re-keying should happen (i.e. you can set-up the OCSP responder to renew its keys and certificates when its current certificate is about to expire). Manual re-keying allows you to trigger the renewal by sending a GET request to the OCSP responder (with the necessary parameters). Manual re-keying is useful when a greater control on re-keying periods is desired. Since manual re-keying can be done with external tools (like wget or curl), cron jobs can be set-up to trigger it at the desired time.

Both automatic and manual re-keying require that EJBCA CA web-service URL is defined. The web-service URL should point to the EJBCA CA server which has issued the certificates for the OCSP responder. If the URL is not defined, the re-keying won't be enabled.

OCSP responder acts as a registration authority when renewing keys with the EJBCA CA. The OCSP responder uses an AuthenticationKeyBinding for SSL client authentication to the CA's web-service.

Since OCSP responder is acting as a registration authority, its certificate for authenticating (in the AuthenticationKeyBinding) to the EJBCA CA web-service must have the key usage set to "Digital Signature" and "keyEncipherment" and extended key usage set to "Client Authentication". It is also necessary to set-up the appropriate access rules on the CA side (either by creating a new RA role, or using an existing one). The role for the OCSP responder should have the right to view and edit the end entities (at least for all of the CA's issuing the certificates for the OCSP responder, as well as for certificate profiles used by the OCSP responder's certificates).

For manual re-keying the GET request should contain the parameters "renewSigner" and "password". The "renewSigner" parameter can be used to specify which OCSP keys should be renewed. It can be either set to "all", which will renew all of the OCSP signer keys, or to a specific OCSP responder certificate subject DN. The parameter specified here should be the OCSP's subject DN, not the issuer's. Password should be configured in the ocsp.properties file. If the password is not set, manual re-keying will not be enabled.

Manual re-keying can further be limited by specifying the allowed originating IP addresses for the requests. By default the re-keying is allowed only from the localhost (127.0.0.1) address. Allowed IP addresses are configured in the ocsp.properties configuration file, and multiple addresses can be provided by separating them with a semicolon (;).

The following two examples demonstrate the manual triggering of re-keying on the OCSP responder. The first example triggers the re-keying for all of the OCSP signer keys, while the second one will trigger rekeying for a specific signer (the one matching the subject DN of "CN=OCSP REsponder 123,O=PrimeKey Solutions AB,C=SE"):

wget http://va.example.com:8080/ejbca/publicweb/status/ocsp?renewSigner=all\&password=foobar123
wget http://va.example.com:8080/ejbca/publicweb/status/ocsp?renewSigner=CN=OCSP\ Responder\ 123,\ O=PrimeKey\ Solutions\ AB,\ C=SE\&password=foobar123

If a specific OCSP subject DN is provided, the OCSP responder will look amongst its keystores/HSM slots for matching certificate and its associated keys. If the specified subject DN is not found, an ERROR message will be output to the server log files specifying the searched subject DN and the available subject DN's. If you're having problems with manual re-keying when providing a specific subject DN, make sure to check the logs and verify that the proper subject DN was specified for the "renewSigner" parameter. Ordering of subject components matters. You can also copy/paste the subject DN from the log to make sure the spelling and ordering is right (i.e. that it matches with what can be found on-disk or in HSM).

For re-keying to work, the OCSP signer certificates need to be issued to separate end entities on the EJBCA CA (i.e. you can't re-use the same end entity for multiple OCSP signer certificates for different CA's).

Be aware that the re-keying operation has not been tested on all of the application servers. Some of the application servers may have problematic client web-service implementations.

Error handling

If there is an error publishing to the OCSP database, the OCSP responder will be out of sync with the CA. It is very important to re-synchronize the databases in that case. Read about error handling and synchronization of the database in the VA installation guide.

Running several responders

Additional OCSP DataSources for publishers on the CA have to be added manually. The easiest way to do this on JBoss is to clone the initially deployed OCSP DataSource JBOSS_HOME/server/default/deploy/ocsp-ds.xml to JBOSS_HOME/server/default/deploy/ocsp2-ds.xml and change

      <jndi-name>OcspDS</jndi-name>
      <connection-url>jdbc:mysql://ocsp1.domain.org:3306/ejbca</connection-url>

to

      <jndi-name>Ocsp2DS</jndi-name>
      <connection-url>jdbc:mysql://ocsp2.domain.org:3306/ejbca</connection-url>

and configure an additional publisher to use this new DataSource 'java:/Ocsp2DS'.

An alternative approach for MySQL users is to use the tools for database replication. Either you could replicate CertificateData from you master EJBCA database to slave-responders or you could publish to a master OCSP responders database that in turn is replicated to the other responders. How to do it is described in the mysql documentation. Depending on which which version you are using please read one of the followings: MySQL 5.0 Replication Howto MySQL 5.1 Replication Howto

Adding additional responders in a live environment

There is no automated way of pushing all the certificates that has been published to existing OCSP responders. To duplicate an existing "source" OCSP database to a "target" OCSP database:

1. To create the tables in the target OCSP, start JBoss AS with OCSP deployed for the first time (and then stop the server before doing the next step).
2. Add an additional DataSource for the target OCSP responder in EJBCA.
3. Configure a new ValidationAuthorityPublisher (Enterprise feature only) in EJCBA that uses the target OCSP DataSource. Chose to only publish to queue to accumulate all changes during the cloning.
4. Wait one hour and check that there is nothing in the publisher-queue of the source OCSP that is older than one hour.
5. Do a MySQL dump from the source database to the target database or use the ClientToolBox DBCOPY-command.
6. When the copy operation has finished, configure a new Republisher Service for the target's OCSP Publisher.
7. Make sure that the queue that built up during the copy operation is now published to the target OCSP.
8. Run the monitoring tool (ClientToolBox OCSPMon) to verify that the new OCSP is in sync.
9. Start the new OCSP node and add it to the pool of OCSPs in your load balancer.

Audit and Account Logging

There are three types of logs that can be generated by the OCSP responder.

1. The OCSP service logs using Log4j to the JBoss server.log. The JBoss server log is located in JBOSS_HOME/server/default/log/server.log and the logging is configured in JBOSS_HOME/server/default/conf/jboss-log4j.xml.

2. The OCSP transaction log can be used to log various information about ocsp-requests. Transaction logging logs summary lines for all OCSP request/responses, which can be used for charging clients if you are running a commercial OCSP service.
To turn on transaction logs logs, copy ocsp.properties.sample to ocsp.properties and change:

#ocsp.trx-log = false

to

ocsp.trx-log = true

then uncomment the other lines below that starts with ocsp.trx-log. Change the ocsp.trx-log-log-date line if you want to change how the time recorded in logging should be output. The value should be on the same format as for javas DateFormat, information on valid configurations can be found here.

ocsp.trx-log-log-date = yyyy-MM-dd:HH:mm:ss

ocsp.trx-log-pattern is a pattern for use with ocsp.audit-order to replace constants with values during logging For most purposes you will not need to change this string.

Use ocsp.trx-log-order to specify what information should be logged and in what order. You can also configure what characters you want in between. If you want your log to display all of the values available you only have to un-comment it.

Available values for the transaction log are:

LOG_ID, An integer identifying that starts from 1 and is increased for every received request.
SESSION_ID A random 32 Byte long String generated when the OCSP-responder is started.
STATUS, The status of the OCSP-Request. SUCCESSFUL = 0;MALFORMED_REQUEST = 1;INTERNAL_ERROR = 2;TRY_LATER = 3;SIG_REQUIRED = 5;UNAUTHORIZED = 6;
CLIENT_IP, IP of the client making the request.
REQ_NAME, The BC normalized Distinguished Name of the client making the request.
REQ_NAME_RAW, The unnormalized Distinguished Name of the client making the request.
SIGN_ISSUER_NAME_DN, The BC normalized issuer Distinguished Name of the certificate used to sign the request.
SIGN_SUBJECT_NAME, The BC normalized Subject Distinguished Name of the certificate used to sign the request.
SIGN_SERIAL_NO, Certificate serial number of the certificate used to sign the request.
NUM_CERT_ID, The number of certificates to check revocation status for.
ISSUER_NAME_DN, The BC normalized issuer Distinguished Name of the requested certificate.
ISSUER_NAME_DN_RAW, The unnormalized issuer Distinguished Name of the requested certificate.
ISSUER_NAME_HASH, SHA1 hash of the issuer DN.
ISSUER_KEY, The public key of the issuer of a requested certificate.
DIGEST_ALGOR, Algorithm used by requested certificate to hash issuer key and issuer name.
SERIAL_NO, Serial number of the a requested certificate.
CERT_STATUS, The requested certificate revocation status.
REPLY_TIME, The time measured between when the request is received by the responder and when the response is sent. This time includes the time it takes to read the request bytes.
PROCESS_TIME, The time measured between when the request has been read by the responder and when the response is sent. This time starts after the request bytes have been read.
CERT_PROFILE_ID, The integer identifier of the certificate profile that was used to issue the requested certificate.
FORWARDED_FOR, The HTTP X-Forwarded-For header value.

3. The OCSP audit log logs entire requests and responses. This can be useful when requests and responses are signed because the information can be used to verify requests and responses afterwards. Audit logging is configured in the same way as transaction logging.
Valid values for audit logging are:

LOG_ID, An integer identifying that starts from 1 and is increased for every received request.
SESSION_ID A random 32 Byte long String generated when the OCSP-responder is started.
OCSPREQUEST, The (hex encoded) byte[] ocsp-request that came with the http-request.
OCSPRESPONSE, The (hex encoded) byte[] ocsp-response that was included in the http-response.

Note that LOG_ID are of the same value in both trx log and audit log for any request. This means they can be cross referenced. You can retrieve information from the transaction log and verify that the information is valid by using the audit Log.

   <appender name="OCSPTRANSACTION" class="org.jboss.logging.appender.RollingFileAppender">
     <errorHandler class="org.jboss.logging.util.OnlyOnceErrorHandler"/>
     <param name="File" value="${jboss.server.log.dir}/transactions.log"/>
     <param name="Append" value="false"/>
     <param name="MaxFileSize" value="500KB"/>
     <param name="MaxBackupIndex" value="1"/>
     <layout class="org.apache.log4j.PatternLayout">
       <param name="ConversionPattern" value="%d %-5p [%c] %m%n"/>
     </layout>	    
   </appender>

   <appender name="OCSPAUDIT" class="org.jboss.logging.appender.RollingFileAppender">
     <errorHandler class="org.jboss.logging.util.OnlyOnceErrorHandler"/>
     <param name="File" value="${jboss.server.log.dir}/audit.log"/>
     <param name="Append" value="false"/>
     <param name="MaxFileSize" value="500KB"/>
     <param name="MaxBackupIndex" value="1"/>
     <layout class="org.apache.log4j.PatternLayout">
       <param name="ConversionPattern" value="%d %-5p [%c] %m%n"/>
     </layout>	    
   </appender>

   <category name="org.cesecore.certificates.ocsp.logging.TransactionLogger">
         <priority value="DEBUG"/>
         <appender-ref ref="OCSPTRANSACTION"/>
   </category>

   <category name="org.cesecore.certificates.ocsp.logging.AuditLogger">
         <priority value="DEBUG"/>
         <appender-ref ref="OCSPAUDIT"/>
   </category>   
  

<periodic-rotating-file-handler name="OCSPTRANSACTION" autoflush="true">
    <formatter>
        <pattern-formatter pattern="%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%E%n"/>
    </formatter>
    <file relative-to="jboss.server.log.dir" path="transactions.log"/>
    <suffix value=".yyyy-MM-dd"/>
    <append value="true"/>
</periodic-rotating-file-handler>
<periodic-rotating-file-handler name="OCSPAUDIT" autoflush="true">
    <formatter>
        <pattern-formatter pattern="%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%E%n"/>
    </formatter>
    <file relative-to="jboss.server.log.dir" path="audit.log"/>
    <suffix value=".yyyy-MM-dd"/>
    <append value="true"/>
</periodic-rotating-file-handler>
<logger category="org.cesecore.certificates.ocsp.logging.TransactionLogger" use-parent-handlers="false">
    <level name="DEBUG"/>
    <handlers>
        <handler name="OCSPTRANSACTION"/>
    </handlers>
</logger>
<logger category="org.cesecore.certificates.ocsp.logging.AuditLogger" use-parent-handlers="false">
    <level name="DEBUG"/>
    <handlers>
        <handler name="OCSPAUDIT"/>
    </handlers>
</logger>
  

ant jbosslog4jsafer

ant ejbca.ear
ant deployear

<appender name="OCSPTRANSACTION" class="org.cesecore.util.log.SaferDailyRollingFileAppender">
	<errorHandler class="org.cesecore.util.log.ProbableErrorHandler" />
	<param name="File" value="${jboss.server.log.dir}/transactions.log" />
	<param name="Append" value="true" />
	
       <!-- Rollover at midnight each day -->
	<param name="DatePattern" value="'.'yyyy-MM-dd" />
	<layout class="org.apache.log4j.PatternLayout">
        <!-- The default pattern: Date Priority [Category] Message\n -->
		<param name="ConversionPattern" value="%d %-5p [%c] %m%n" />
	</layout>
</appender>
<appender name="OCSPAUDIT" class="org.cesecore.util.log.SaferDailyRollingFileAppender">
	<errorHandler class="org.cesecore.util.log.ProbableErrorHandler" />
	<param name="File" value="${jboss.server.log.dir}/audit.log" />
	<param name="Append" value="true" />

    <!-- Rollover at midnight each day -->
	<param name="DatePattern" value="'.'yyyy-MM-dd" />
	<layout class="org.apache.log4j.PatternLayout">
        <!-- The default pattern: Date Priority [Category] Message\n -->
		<param name="ConversionPattern" value="%d %-5p [%c] %m%n" />
	</layout>
</appender>
	
<logger name="org.cesecore.certificates.ocsp.logging.TransactionLogger">
	<level value="DEBUG" />
	<appender-ref ref="OCSPTRANSACTION" />
</logger>
<logger name="org.cesecore.certificates.ocsp.logging.AuditLogger">
	<level value="DEBUG" />
	<appender-ref ref="OCSPAUDIT" />
</logger>  

OCSP GET

The GET OCSP request is defined in RFC 6960 (and RFC2560) A.1 as: 'GET {url}/{url-encoding of base-64 encoding of the DER encoding of the OCSPRequest}'. A base64-encoded request can contain the reserved characters '+', '/' and '=', but will be handled correctly both in their %-escaped and original form by the responder, since it's unclear if they do conflict as defined in RFC 2396 2.2.

Not all web-product handles the encoded '/' (%2F) nicely. JBoss/Tomcat has to be started with -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true added to JAVA_OPT in JBOSS_HOME/bin/run.conf. On Glassfish this JVM option is configured under Application Server Settings.

Responses with longer validity and caching

RFC 6960 (and RFC2560) defines thisUpdate, nextUpdate and producedAt. producedAt is always included in the response and is the time the response was created. thisUpdate and nextUpdate is enabled by configuring 'ocsp.untilNextUpdate' in ocsp.properties or in the OcspKeyBinding. thisUpdate will be the time a singleResponse is embedded in the main response and nextUpdate will be 'untilNextUpdate' seconds later than thisUpdate. This enables clients that supports this feature to re-use a valid response and decrease to load on the OCSP-responder.

RFC 5019 defines how to use HTTP cache headers as defined in RFC 2616 for OCSP HTTP GET requests. By using the headers Last-Modified, Expires, max-age and Date, less intelligent nextwork component like HTTP caches can cache respones. This enables re-use of responses to decrease the load on the OCSP-responder and can shorten reponse times by deploying caches closer to the actual OCSP consumers. HTTP cache headers is enabled by configuring configuring 'ocsp.maxAge' in ocsp.properties or in the OcspKeyBinding.

When using RFC 5019 style HTTP headers, JBoss users should be aware that the Date header is overwritten with a cached value. Since generating the Date-string is computationally heavy for regular small GET requests, it is generated about once per second. So a response will have a Last-Modified that is one second in the future from Date from time to time.

A regular Apache HTTP server can be used for caching requests, load-balancing and dropping some unwanted requests:

<VirtualHost *:80>
        # Use as much memory as possible for the cache (in 1 kB blocks)
        # 1GB of memory at ~2kB/ocsp request would hold about 500000 different requests
        CacheEnable mem /
        MCacheSize 1048576
        MCacheMaxObjectCount 1000000
        MCacheMinObjectSize 1
        MCacheMaxObjectSize 4096

        # Using disk-cache will allow a much larger pool of cached entires and the operation system
        # will cache those files, but you are responsible for cleaning up old cache-entries using
        # the "htcacheclean" tool. A disk cache will also live through a server restart.
        # The user running apache has to have read/write access to "/var/cache/ocsp".
        #CacheEnable disk /
        #CacheRoot /var/cache/ocsp

        # Ignore requests for uncached responses.. this will protect the OCSP from
        # DOS attacks using "Cache-Control: no-cache" or "Pragma: no-cache"
        CacheIgnoreCacheControl On

        ProxyRequests Off

        <Location>
                # Everybody is welcome here..
                Allow from all
                Order allow,deny

                # ..or just those from networks that is supposed to use the service
                #Deny from all
                #Order deny,allow
                #allow from 127.
                #allow from 172.16.212.1

                ProxyPassReverse balancer://mycluster-kerb/
        </Location>

        # Proxy requests to OCSP instances (only one machine currently configured)
        <Proxy balancer://mycluster-kerb>
                # proxy_ajp has to be enabled for ajp-proxying
                BalancerMember ajp://127.0.0.1:8009/ejbca/publicweb/status/ocsp
                # proxy_http has to be enabled for http-proxying
                #BalancerMember http://ocsp2.domain.org:8080/ejbca/publicweb/status/ocsp
                #BalancerMember http://ocsp3.domain.org:8080/ejbca/publicweb/status/ocsp
        </Proxy>

        # We only want RFC 5019 compliant URLs to be forwarded to the OCSP, the rest
        # should get a "404 Not found" or "414 Request-URI Too Large."
        LimitRequestLine 257
        RewriteEngine On
        RewriteCond %{REQUEST_METHOD} get [NC]
        RewriteRule ^/([a-zA-Z0-9+=/]+)$ balancer://mycluster-kerb/$1 [P,L]

        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel debug
        CustomLog /var/log/apache2/access.log combined
        ErrorLog /var/log/apache2/error.log
</VirtualHost>