Monday, 26 May 2014

AWS deployment options - Part 3 - mod_proxy with autoscaling

Due to the limitation with mod_jk for autoscaling configurations, we took a look at mod_proxy


Instead of using AJP (mod_jk) protocol we can use the HTTP protocol and with this approach we do not use Apache for the actual load balancing or session stickiness. Instead all requests from the Apache server are forwarded to the application tier ELB (say it runs at HTTP port 8080) and the ELB does load balancing as well as maintains session stickiness.

The Apache httpd.conf configuration for the same is mentioned below.





LoadModule proxy_module       modules/mod_proxy.so
LoadModule proxy_http_module  modules/mod_proxy_http.so

ProxyPass /img !
ProxyPass /images !
ProxyPass /css !
ProxyPass /media !

ProxyPass / http://internal-RCCL-AppTier-ELB-722360436.us-east-1.elb.amazonaws.com:8080/
ProxyPassReverse / http://internal-RCCL-AppTier-ELB-722360436.us-east-1.elb.amazonaws.com:8080/







§Pros:

ELB is able to maintain sticky sessions using it’s own cookie named AWSELB

§Cons:

When we need to run multiple Tomcat JVMs on the same instance, this approach fails. For eg: If we have 6 Tomcats distributed as Server 1 on ports 8080, 8085, 8090 and Server 2 on ports 8080, 8085, 8090. In this case, on Apache we need to switch to mod_proxy_balancer and Apache will load balance across 3 ELBs running on 8080, 8085 and 8090 respectively.

In such a scenario, session stickyness fails again as it again requires the same combination of unique node names (jvmRoutes) for each Tomcat instances and these need to be pre-defined on Apache as well. So we face the same issue on mod_jk See http://stackoverflow.com/questions/9393163/apache-tomcat-problems-with-sticky-sessions-and-load-balancing
 
So if you need a single Tomcat per JVM then this is the best solution.

For multiple JVMs there is an alternate recommendation given to us by Amazon support, and that is to use one ELB per port and set up the tomcat tier with multiple ELBs as shown below.

 
This will work provided you allocate the architecture and resources in the right manner.

 


Friday, 23 May 2014

AWS deployment options - Part 2 - mod_jk with autoscaling


 
If the application requires session stickiness at the Tomcat tier, i.e. if some user details are stored in the HTTP Session and you don't enable session replication across Tomcats. The solution is that the Elastic Load Balancer (ELB) maintains the sticky session using it’s own cookie called AWSELB. For ELB sticky session configuration see http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.managing.elb.html
In case there is an external ELB (Apache tier) with cookies stickiness set, it will over-ride the internal ELB (Tomcat tier). The stickiness cookie is always named AWSELB, hence the external ELB will overwrite the cookie from the internal ELB. One option here is to set the Tomcat tier ELB with "Application Generated Cookie Stickiness" with value JSESSIONID so it will follow the Application cookie and maintain sticky sessions.
 

In case sticky session is not enabled on the application tier, and the user is forwarded to a Tomcat server where the session is not present, the user’s session information will be lost

There are various approaches available to maintain session stickiness, and these have been discussed below.
 
Option 1: Use mod_jk to get Apache to direct to the correct Tomcat

Option 2: Use mod_proxy

Option 3: No session stickiness, keep it stateless (which is what Amazon recommends)

The mod_jk implementation looks like this
 
 
The Apache web server can maintain session stickiness along with load balancing i.e. The requests from the Apache server to the Tomcat server are load balanced along with maintaining session stickiness, by Apache server itself .(The application ELB is bypassed).
In this approach the Apache server needs to know the list of all possible Tomcat IPs.
When we set up autoscaling, Tomcat can start up new instances on any IP within the subnet. That's the way AWS works.
Hence this is not feasible as when we restart the AWS servers the IPs might change.
 
 
 

 Mod JK Configuration

The file “httpd.conf” at the location “/www/Apache/conf” needs to have the following entries. These entries indicate that the files from folders “img”, “images”, ”css” and “media” will be served locally where as rest of the requests will be forwarded to the load balancer “lb”.



LoadModule jk_module modules/mod_jk-1.2.30-httpd-2.2.X.so
JkWorkersFile conf/workers.properties
JkLogFile logs/mod_jk.log
JkLogLevel error
JkShmFile logs/mod_jk.shm
 
JKMount /status jkstatus
 
JkAutoAlias /apps/Tomcat/apache-tomcat-7.0.42/webapps
 
JkMount  /* lb
JkUnMount /img/* lb
JkUnMount /images/* lb
JkUnMount /css/* lb
JkUnMount /media/* lb
 
 




 
 
 
 
 
 
 
 
 
 
 
 

Also the file “worker.properties” will also need to be modified as indicated below.



worker.list=lb,jkstatus
 
#node 1
worker.node1.type=ajp13
worker.node1.port=8009
worker.node1.host=172.23.0.36
worker.node1.lbfactor=1
worker.node1.cachesize=10
 
#node 2
worker.node2.type=ajp13
worker.node2.port=8009
worker.node2.host=172.23.0.38
worker.node2.lbfactor=1
worker.node2.cachesize=10
 
#node 3
worker.node3.type=ajp13
worker.node3.port=8009
worker.node3.host=172.23.1.36
worker.node3.lbfactor=1
worker.node3.cachesize=10




 


 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 This maps to the Tomcat server.xml for each JVM
 



 
§Pros:
Has very good support for failure detection and switchover to alternate Tomcat server.
Use ELB for health checks and not for load balancing
§Cons:
The session stickyness is using cookies like this jsessionid=AB327CD87667461357C95742BB9974B1.node1 which is maintained via configuration in Apache workers.properties. In this all Tomcat IP addresses and ports must be mentioned along with a logical name for each Tomcat (node1 in this case)
That logical name
must be the same as entered in each individual Tomcat server.xml as it’s “jvmRouteas well. Thus we will have Tomcat node1, node2 .. nodeN
This affects autoscaling because Tomcats will be a new instance autoscaled on any available IP (from a set of IPs) and it’s not possible to hardcode a “nodeX” in Tomcat when it is launched via autoscaling
For this reason we next take a look at mod_proxy


 
 
 

Thursday, 22 May 2014

Amazon AWS deployment options for Apache web server to Tomcat connectivity



The cloud proposition is attractive because of the ability to scale up or down to meet workload demand almost instantaneously and to utilize efficiencies in the services delivery model. The rationale of using and paying for only what you need, when you need it is a very compelling business justification.

 

This post and the next few ones describes the various options and features available when a web application is deployed on the AWS cloud infrastructure using Apache HTTP Web Server and Apache Tomcat application server. This document deals more with options available to maintain sticky sessions and application persistence.
 
This is based on a recent project where we are deploying with Apache 2.4 and Tomcat 7.0.x
 
First some basic terms
 

 AWS Glossary

 
·                     Amazon EC2 is hosted in multiple locations world-wide. These locations are composed of Regions and Zones. Each region is a separate geographic area.
Each region has multiple, isolated locations known as Availability Zones. This supports high availability of the applications and workloads can be balanced across both availability zones.
·                     To maintain a high-level of security the different tiers such as Web, App and Database are divided into separate subnets and "Security Groups", which are roughly analogous to firewalls.  This is done through Amazon VPC, a logically isolated section of the AWS Cloud where customers can launch AWS resources into a virtual network which they define. Capability to control inbound and outbound access to and from individual subnets using network access control lists, i.e. stateless firewall.  
·                     Load balancer (Elastic Load Balancer)
Elastic Load Balancing automatically distributes incoming application traffic across multiple Amazon EC2 instances. Elastic Load Balancing also detects unhealthy instances and automatically reroutes traffic to healthy instances until the unhealthy instances have been restored. Elastic Load Balancing can also be enabled within a single or multiple Availability Zones for more consistent application performance.
ELB also maintains Sticky sessions in case the application requires it.

 
Logical View
 
 
 

The application is hosted across two Zones within a region.
There is a Web tier ELB to load balance across the Apache servers as well as an Application tier ELB to load balance requests across the Tomcat servers. Cross-zone load balancing is optional and can be configured as required.
The requests from the Apache server to the Tomcat server can also be load balanced by Apache server itself (i.e. the application tier ELB is bypassed and not used). But this is not a preferred approach for AWS.
Hence both the application as well as the web tier ELB’s are used for load balancing.
In the next post I will explain how to use either mod_jk or mod_proxy to achieve the load balancing plus sticky sessions.
 

 

Saturday, 15 March 2014

Now Playing - March 2014


Music albums that I've liked over the last few months:

Radical Face - love this album The Family Tree: The Branches - reminds me of Bon Iver and The Verve
The National - Trouble Will Find Me
Of Monsters and Men
The Secret Life of Walter Mitty - OST - Jose Gonzalez has a superb song titled Step Outside
Pearl Jam - Lightning Bolt

Aside on Bon Iver's Holocene - which was part of the OST for Cameron Crowe's "We Bought a Zoo" - the lyrics and tune remind you of vast open white wilderness (ignore the geological meaning) - which reminds me of what a Vanilla Sky would look like - which is another of Cameron Crowe's films.

Wednesday, 15 January 2014

Solution for Samsung Galaxy Hanging/Freezing Screen - Samsung Galaxy Core Duos GT-I8262

My Galaxy Core hangs all the time. I uninstalled all downloaded apps and also I did a factory reset, but that also didn’t help.

Some googling led me onto this post on XDA developers. If you want more info (and know more about the risks), go to the XDA forum and read the whole thing.

Temporary Fix

So the steps are install Dummy File Generator (DFG) on your phone. This app can populate your entire internal memory and SD-card memory with dummy data. I completely filled the internal memory with that tool. After that go back to the tool and choose "Delete" of all the dummy files.

You will get a warning message that memory is nearly full. Whatever you do - do not reboot in between this, because the phone won't reboot after that.

If the DFG hangs when trying to delete dummy files, then go to Application Manager > Dummy File Generator > Clear Data.

After this there will still be the odd freeze but not as severe as before (may be once a day).

More Reading:
http://androidforums.com/samsung-galaxy-s3/749952-solved-moment-s3-apps-crash-phone-freezes-everyday-factory-reset-already.html
http://forums.androidcentral.com/samsung-galaxy-siii-i9300/269096-s3-hanging-freezing-problem-solved.html

Better Fix

I took the phone to the Samsung mobile centre, they told me the problem actually is introduced after you do a software upgrade directly over wi-fi or network. They recommend downgrading to Factory version and they do the software upgrade for you and in 90% of the cases, this fixes the hanging problem.

Thursday, 13 June 2013

Maven build - java.lang.IllegalArgumentException: The argument does not represent an annotation type


When you run a Maven build and get this error

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.
0:compile (default-compile) on project celebrity-cda-web: Fatal error compiling:
 java.lang.IllegalArgumentException: The argument does not represent an annotati
on type: XStreamImplicit -> [Help 1]
[ERROR]


In this case Maven cannot find the class XStreamImplicit

The problem is that a <dependency> which was earlier defined in the pom.xml has now been deleted or the Maven repository is not defined correctly for that dependency.


 

Sunday, 31 March 2013

HTC Wildfire Reset to Stock ROM


If you’re on cyanogenmod or some other rooted ROM and you’re missing the smoothness of HTC Sense J , worry not.

 If you didn’t take a Nandroid Backup of the stock ROM when rooting your phone, you’ll have to download and install the correct version of the stock HTC ROM for your model and region.

Some links to these ROMs are at


Generally, RUU's are named as


RUU_Buzz____Radio__release_.exe
 

 Note the EXE versions will UNROOT your phone and leave you with an original HTC ROM in case you need to claim a warranty fix on your phone.

 I prefer the ZIP versions which keep the phone with ROOT and you can then use Root Uninstaller from the Play Store to disable any system apps which you don’t want to use.

 For Wildfire S Asia India RUU this is the ZIP version of the ROM


 And for plain Wildfire i.e. A3333 use the download from here

In case the link is broken Search for “Buzz_Froyo_hTC_Asia_India_2.25.720.1-dannejo.zip”

Make sure you download the valid ROM for your REGION, and for your Device Model. The above link is only for ASIA_INDIA

Before you do anything else,

Take a Nandroid Backup of your working ROM.

If you use the wrong ROM, you will be stuck waiting on the HTC Logo screen for like ever.

If your phone is bricked in that manner, remove and then reinsert the battery and try to get into Recovery mode into one of the earlier Nandroid backups.

 Installation Method:

Take a Nandroid Backup of your Cyanogenmod ROM in case you want to revert to it later
  • Copy the ZIP to root of your SD card
  • Wipe Dalvik Cache
  • Factory Reset
  • Then flash the zip from SD card

First reboot takes a while – all else good after that