Apache configuration for Internet Apps using JBoss

Apache configuration for Internet Apps using JBoss

Downloaded Apache 2.2.6 and mod_jk-1.2.25-httpd-2.2.4.so

Compiled apache with:

./configure --prefix=/apps/apache2 --enable-mods-shared=all --enable-cache --enable-log-forensic --enable-mime-magic=shared --enable-usertrack=shared --enable-ssl --enable-dav=shared --enable-module=rewrite --enable-rewrite=shared --enable-speling=shared --enable-mod_auth=shared --enable-mod_access=shared --enable-mod_auth_anon=shared --enable-module=proxy --enable-mod_proxy=shared --enable-proxy=shared --enable-module=mod_proxy_html --enable-mod_proxy_html=shared --enable-mod_proxy_http=shared

make

make install

copied mod_jk-1.2.25-httpd-2.2.4.so to /apps/apache2/modules/mod_jk.so

Tarred up the compiled httpd-2.2.6 folder and uploaded to the yum server

New Apache doesn't support the mod_wl_20 that I have and I'm not using weblogic clustering on the old servers so I'll just convert all the MatchExpression statements to ProxyPass and ProxyPassReverse until the apps are finished being ported to JBoss.

Also, will be removing SSL from apache and using Foundry ServerIron to handle SSL and load balancing in near future.

I created a file called redirect.txt in the vhosts.d folder with the following:

# Rules to forward weblogic applications through the secure server this sends everything from http://....../app1 to https://..../app1/
RedirectMatch temp (/app1) https://webserver.pvalentino.org/app1/
RedirectMatch temp (/app2) https://webserver.pvalentino.org/app2/
RedirectMatch temp (/app3) https://webserver.pvalentino.org/app3/

Don't ask me why but a standard redirect did not work when using SSL for me....

Then I include this redirect in my standard html virtual host - this is a single file in my vhosts.d directory named www.pvalentino.org( i address the weblogic proxying in the next step ):

NameVirtualHost 10.2.5.40:80

<VirtualHost 10.10.15.140:80>
ServerName webserver.pvalentino.org:80
ServerAlias webserver.pvalentino.org *.pvalentino.org
ServerAdmin webmaster@pvalentino.org
DocumentRoot /apps/apache2/htdocs/www.pvalentino.org
ErrorLog /log/www.pvalentino.org-error_log
CustomLog /log/www.pvalentino.org-access_log combined
Include conf/vhosts.d/redirect.txt

<Directory "/apps/apache2/htdocs/www.pvalentino.org">
AllowOverride None
AddOutputFilter INCLUDES .htm
AddOutputFilter INCLUDES .html
Options +Includes -Indexes
</Directory>

</VirtualHost>

Now for my ssl virtual host i have a file called ssl.pvalentino.org in the same vhosts.d folder

<VirtualHost 10.2.5.40:443>
ServerName webserver.pvalentino.org:443
ServerAlias webtest.pvalentino.org:443
ServerAdmin webmaster@pvalentino.org
DocumentRoot /apps/apache2/htdocs/www.pvalentino.org
ErrorLog /log/webserver.pvalentino.org-error_log
CustomLog /log/webserver.pvalentino.org-access_log combined
Include conf/vhosts.d/weblogic.txt
#
<Directory "/apps/apache2/htdocs/www.pvalentino.org">
AllowOverride None
AddOutputFilter INCLUDES .htm
AddOutputFilter INCLUDES .html
Options +Includes -Indexes FollowSymLinks
</Directory>

SSLEngine on
SSLCertificateFile /data/web/certs/www.pvalentino.org.crt
SSLCertificateKeyFile /data/web/certs/www.pvalentino.org.key

</VirtualHost>

And for the ProxyPass to weblogic you will see i have an include for weblogic.txt above which contains:

# App1
ProxyPass /app1 http://wlstest.pvalentino.lan:8087/app1
ProxyPassReverse /app1 http://wlstest.pvalentino.lan:8087/app1
# App2
ProxyPass /app2 http://wlstest.pvalentino.lan:8088/app2
ProxyPassReverse /app2 http://wlstest.pvalentino.lan:8088/app2

Add conf and conf.d and vhosts.d directory include entries to /apps/apache2/conf/httpd.conf:

 # Include mod_jk and other configuration files

Include conf.d/*.conf

Include conf/vhosts.d/*.conf

Include conf/mod_jk.conf

Created ssl.conf in conf.d folder with:

Listen 443

AddType application/x-x509-ca-cert .crt
AddType application/x-pkcs7-crl    .crl

SSLPassPhraseDialog  builtin

SSLSessionCache         shmcb:/var/cache/mod_ssl/scache(512000)
SSLSessionCacheTimeout  300

SSLMutex default

SSLRandomSeed startup file:/dev/urandom  256
SSLRandomSeed connect builtin

SSLCryptoDevice builtin

Added rewrite.conf to conf.d:

# Prevent HTTP TRACE/TRACK requests (security)
RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK)
RewriteRule .* - [F]

Also, change MaxClients to 80% of = st1 JBoss AS maxThreads in httpd.conf(in this case 400)

MaxClients       400

Update /etc/httpd/conf.d/workers.properties as follows:

# Define list of workers that will be used

# for mapping requests

worker.list=loadbalancer,loadbalancer2,jkstatus

# Define Node1

# modify the host as your host IP or DNS name.

worker.node1.port=8009

worker.node1.host=10.10.14.17

worker.node1.type=ajp13

worker.node1.connection_pool_timeout=600
worker.node1.socket_keepalive=true
worker.node1.lbfactor=3

worker.node1.socket_keepalive=1

# Inet

worker.inet1.port=9009

worker.inet1.host=x.x.x.x or preferably name

worker.inet1.type=ajp13

worker.inet1.connection_pool_timeout=600
worker.inet1.socket_keepalive=true
worker.inet1.lbfactor=3

worker.inet1.socket_keepalive=1

worker.inet2.port=10009

worker.inet2.host=x.x.x.x or preferably name

worker.inet2.type=ajp13

worker.inet2.connection_pool_timeout=600
worker.inet2.socket_keepalive=true
worker.inet21.lbfactor=3

worker.inet2.socket_keepalive=1

# Load-balancer 1

worker.loadbalancer.type=lb

worker.loadbalancer.balance_workers=node1

worker.loadbalancer.sticky_session=1

# Load-balancer 2

worker.loadbalancer2.type=lb

worker.loadbalancer2.balance_workers=inet1, inet2

worker.loadbalancer2.sticky_session=1

# Status worker

worker.jkstatus.type=status

Update /apps/apache2/conf/uriworkermap.properties as follows:

/app=loadbalancer

/app/*=loadbalancer

/appws=loadbalancer

/appws/*=loadbalancer

/app1=loadbalancer2

/app1/*=loadbalancer2

/app2=loadbalancer2

/app2/*=loadbalancer2

/jkmanager=jkstatus

Verify that /apps/apache2/conf/mod_jk.conf is configured as follows:

# Specify the filename of the mod_jk lib

LoadModule jk_module modules/mod_jk.so

# Where to find workers.properties

JkWorkersFile conf/workers.properties

# Where to put jk logs

JkLogFile logs/mod_jk.log

# Set the jk log level [debug/error/info]

JkLogLevel debug

# Select the log format

JkLogStampFormat "[%a %b %d %H:%M:%S %Y]"

# JkOptions indicates to send SSK KEY SIZE

JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories

# JkRequestLogFormat

JkRequestLogFormat "%w %V %T"

JkMountFile conf/uriworkermap.properties

# Add shared memory.

# This directive is present with 1.2.10 and

# later versions of mod_jk, and is needed for

# for load balancing to work properly

JkShmFile logs/jk.shm

# Add jkstatus for managing runtime data

<Location /jkstatus/>

JkMount jkstatus

Order deny,allow

Deny from all

Allow from all

</Location>

Paths to server.xml (substitute app1 for actual application names):

/apps/jboss/default/jboss/server/app1/deploy/jboss-web.deployer/server.xml

/apps/jboss/default/jboss/server/all/deploy/jboss-web.deployer/server.xml

/apps/jboss/default/jboss/server/default/deploy/jboss-web.deployer/server.xml

Edit server.xml and add a jvmRoute attribute to the <Engine> element.

!!! Automation note - I automated the part below by changing the source code to use jvmRoute="uniqueid" and then added the following to the deployment script:

JVM_DIR=${PV_SERVER_DIR}/deploy/jboss-web.deployer/server.xml
JVM_ROUTE=`hostname`${PV_SERVER}
sed -i s/uniqueid/${JVM_ROUTE}/g ${JVM_DIR}
j=$?
if (( $j !=0 ))
then
echo "Problems updating jvmRoute in server.xml for ${PV_SERVER}"
echo "Current settings are:"
cat ${JVM_DIR} |grep jvmRoute
fi
# PV Server is defined earlier in the script and basically maps to $1 which is passed by the anthill job depending on which workflow you're running so all #applications automatically get their own unique jvmRoute at deploy time just validate these settings and replace all the worker.inet1 entries with worker.$JVM_ROUTE value

Server inet1:

<Engine name="jboss.web" defaultHost="localhost" jvmRoute="inet1">

... ...

</Engine>

Server inet2:

<Engine name="jboss.web" defaultHost="localhost" jvmRoute="inet2">

... ...

</Engine>

On inet1:

<!-- A AJP 1.3 Connector on port 9009 -->

<Connector port="9009"

address="${jboss.bind.address}"

maxThreads="250"

emptySessionPath="true"

enableLookups="false"

redirectPort="8443"

protocol="AJP/1.3"/>

On inet2:

<!-- A AJP 1.3 Connector on port 10009 -->

<Connector port="10009"

address="${jboss.bind.address}"

maxThreads="250"

emptySessionPath="true"

enableLookups="false"

redirectPort="8443"

protocol="AJP/1.3"/>

Paths for jboss-service.xml (substitute app1 for actual application names):

/apps/jboss/default/server/app1/deploy/jbossws.sar/META-INF/jboss-service.xml

/apps/jboss/default/server/app1/deploy/jboss-web.deployer/META-INF/jboss-service.xml

/apps/jboss/default/server/all/deploy/jbossws.sar/META-INF/jboss-service.xml

/apps/jboss/default/server/all/deploy/jboss-web.deployer/META-INF/jboss-service.xml

/apps/jboss/default/server/app1/conf/jboss-service.xml

/apps/jboss/default/server/all/conf/jboss-service.xml

The jvmRoute attribute must match the name specified for this JBoss AS instance in the workers.properties file. Next, edit the jboss-service.xml files, and set the UserJK attribute to true. Again - did this in source so it is deployed with anthill build

On both inet1 and inet2:

<attribute name="UseJK">true</attribute>