Simple Java Proxy Servlet Using HttpURLConnection

Aim
The aim of this is to create a simple Java Proxy Servlet using HttpURLConnection. About 75 lines of code only. How simple is that!? The java servlet is deployed to the 'root' context of any java container such as Glassfish or Tomcat. You have to configure your web browser proxy settings to point to the host and port where the servlet is deployed. The java source code is available through a link below. Please leave any question or comments at the end and I will endeavour to answer them.

Assumptions
This article assumes that you have Glassfish or Tomcat or an equivalent java container installed and configured. This is a basic proxy and does not cover some features such as https etc.

Versions used in this example
Sofware/Component
Image
Windows XP SP2
N/A
Glassfish
N/A
Java JDK 1.5.0
N/A
Links to these files can be found here

For the purposes of this tutorial the files are stored in the structure below. The ServletProxy directory is the working directory. You can ignore the buil.xml and passwords.txt file as these are not required to compile the example. However, if you have glassfish and ant installed you can use the build.xml file.

ServletProxy
_|_web_bin
_|__|_WEB-INF
_|__|__|_web.xml
_|__|__|_classes
_|_web_src
_|__|_myproxy
_|__|__|_MyProxy.java
_|_build.xml
_|_passwords.txt

You can download the zipped example here.

Create and deploy the Service
  1. Write and save the code below as myproxy/MyProxy.java in the web_src directory. This is the MyProxy servlet class. It recreates the url and uses HttpURLConnection to fetch the data from the requested host. As you can see the servlet request headers are copied to the outgoing HttpURLConnection headers, and incomming HttpURLConnection headers are copied back to the servlet response. This way we don't have to mess around with any headers.
     1. package myproxy;
     2. 
     3. import java.io.*;
     4. import java.util.Enumeration;
     5. import java.util.Iterator;
     6. import java.util.Map;
     7. import java.util.List;
     8. import java.util.logging.*;
     9. 
    10. import javax.servlet.*;
    11. import javax.servlet.http.*;
    12. 
    13. import java.net.URL;
    14. import java.net.HttpURLConnection;
    15. 
    16. public class MyProxy extends HttpServlet {
    17.     
    18.     private ServletContext servletContext;
    19.     private Logger log;
    20.     
    21.     public void init(ServletConfig servletConfig) throws ServletException {
    22.         servletContext = servletConfig.getServletContext();
    23.         log = Logger.getLogger(MyProxy.class.getName());
    24.     }
    25.     
    26.     public void doGet(HttpServletRequest request, HttpServletResponse response){
    27.         doPost(request, response);
    28.     }
    29. 
    30.     public void doPost(HttpServletRequest request, HttpServletResponse response){
    31. 
    32.         BufferedInputStream webToProxyBuf = null;
    33.         BufferedOutputStream proxyToClientBuf = null;
    34.         HttpURLConnection con;
    35.         
    36.         try{
    37.             int statusCode;
    38.             int oneByte;
    39.             String methodName;
    40.             String headerText;
    41.             
    42.             String urlString = request.getRequestURL().toString();
    43.             String queryString = request.getQueryString();
    44.             
    45.             urlString += queryString==null?"":"?"+queryString;
    46.             URL url = new URL(urlString);
    47.             
    48.             log.info("Fetching >"+url.toString());
    49.             
    50.             con =(HttpURLConnection) url.openConnection();
    51.             
    52.             methodName = request.getMethod();
    53.             con.setRequestMethod(methodName);
    54.             con.setDoOutput(true);
    55.             con.setDoInput(true);
    56.             con.setFollowRedirects(false);
    57.             con.setUseCaches(true);
    58. 
    59.             for( Enumeration e = request.getHeaderNames() ; e.hasMoreElements();){
    60.                 String headerName = e.nextElement().toString();
    61.                 con.setRequestProperty(headerName,    request.getHeader(headerName));
    62.             }
    63. 
    64.             con.connect();
    65.             
    66.             if(methodName.equals("POST")){
    67.                 BufferedInputStream clientToProxyBuf = new BufferedInputStream(request.getInputStream());
    68.                 BufferedOutputStream proxyToWebBuf     = new BufferedOutputStream(con.getOutputStream());
    69.                 
    70.                 while ((oneByte = clientToProxyBuf.read()) != -1) 
    71.                     proxyToWebBuf.write(oneByte);
    72.                 
    73.                 proxyToWebBuf.flush();
    74.                 proxyToWebBuf.close();
    75.                 clientToProxyBuf.close();
    76.             }
    77.             
    78.             statusCode = con.getResponseCode();
    79.             response.setStatus(statusCode);
    80.             
    81.             for( Iterator i = con.getHeaderFields().entrySet().iterator() ; i.hasNext() ;){
    82.                 Map.Entry mapEntry = (Map.Entry)i.next();
    83.                 if(mapEntry.getKey()!=null)
    84.                     response.setHeader(mapEntry.getKey().toString(), ((List)mapEntry.getValue()).get(0).toString());
    85.             }
    86.             
    87.             webToProxyBuf = new BufferedInputStream(con.getInputStream());
    88.             proxyToClientBuf = new BufferedOutputStream(response.getOutputStream());
    89. 
    90.             while ((oneByte = webToProxyBuf.read()) != -1) 
    91.                 proxyToClientBuf.write(oneByte);
    92. 
    93.             proxyToClientBuf.flush();
    94.             proxyToClientBuf.close();
    95. 
    96.             webToProxyBuf.close();
    97.             con.disconnect();
    98.             
    99.         }catch(Exception e){
    100.             System.out.println(e.getMessage());
    101.             e.printStackTrace();
    102.         }
    103.         finally{
    104.         }
    105.     }
    106. }
    Hide line numbers

  2. Now cd into the web_src directory and compile the source. Replace the path to Glassfish's lib directory with your local path. This could be lib in tomcat or any servlet container.

    ..ServletProxy\web_src>javac myproxy\MyProxy.java -d ..\web_bin\WEB-INF\classes -extdirs d:\downloads\glassfish\lib

  3. Write the web.xml file and save in under the web_bin/WEB-INF directory.
     1. <?xml version="1.0" encoding="ISO-8859-1"?>
     2. <web-app>
     3.     <servlet>
     4.         <servlet-name>My Proxy</servlet-name>
     5.         <servlet-class>myproxy.MyProxy</servlet-class>
     6.     </servlet>
     7. 
     8.     <servlet-mapping>
     9.         <servlet-name>My Proxy</servlet-name>
    10.         <url-pattern>/</url-pattern>
    11.     </servlet-mapping>
    12. </web-app>
    Hide line numbers

  4. Now create the war file. Cd into the web_bin directory and jar it all up.

    ..ServletProxy\web_bin>jar -cvf proxywala.war *

  5. Finally deploy this war file in the servlet container of your choice. Remember it has to be in the ROOT context! .

    For glassfish you can specify the root context in the admin console. The steps for glassfish are.
    1. Login to the glassfish console
    2. Navigate to Applications -> Web Applications.
    3. Select deploy.
    4. Under "location" navigate to the war file we created proxywala.war.
    5. Under "Context Root" type in "/".
    6. Click Ok.

    For tomcat the steps are
    1. Copy the contents of the web_bin directory. Make sure it's the contents only such as WEB-INF and web.xml etc. Do not copy the web_bin directory itself.
    2. Now paste the contents into the tomcat webapps/ROOT directory.
    3. Restart Tomcat - if required.

Testing the filter
  1. First change your proxy settings to point to the proxy. In this example the servlet is deployed on the localhost at port 80.
    Proxy http://localhost
    proxy Port 80

  2. Now navigate to a some sites in your brower. You will be going to the web from your proxy. This proxy needs direct internet access.

21 comments:

Anonymous said...

You should check if the response code is different than 200/201 and get the errorstream instead of the inputstream

Anonymous said...

This doesn't work for 'https' URLs.

righteous said...

I have stated that this doesn't work with https. However it would be fairly easy to get it to work with https. All you have to do is include another https listener, and your browser settings to point to the new listener for https

Saurabh said...

Is it possible to use this proxy project as an individual project (not in ROOT)? If yes, how is it possible?

righteous said...

Has to be root. Note that we're not actually using a servlet the way it's meant to be used.

Saurabh said...

thanks for your reply sir. Actually this is my college project and want to create something like this "http://www.webproxyserver.net". Can you please help me or suggest me some useful links to develop this kind of web proxy server in Java.

thanks in advance

righteous said...

these web proxies are not real proxies, they are url rewriting scripts. You can search for scripts such as Glype,PHPProxy etc.

I have an introduction to this kind of proxy here.
Simple web proxy

Or visit
codediaries.com

Anonymous said...

I would like to rewrite URL's using this as a starting point. I tried:

webToProxyBufReader = new BufferedReader(new InputStreamReader(con.getInputStream()));
String l = null;
while ((l = webToProxyBufReader.readLine()) != null){
l = rewriteUrls(l);
proxyToClientBuf.write(l.getBytes());
}



But it messes-up images and also seems to hang on fetching javascript files.

Any suggestions?

righteous said...

You need to test for content-type and NOT touch binary stuff such as images and streams etc. You should only modify text.

Rewriting javascript can be tricky as you could break something in the code.

What are you rewriting the urls to?

Anonymous said...

Thanks for the idea on checking content type.

I'm just trying to hitting my own app server internally (it might be on http:\\appservint.local:9043) but "real" users need to go to the web server https:\\external.com. So all the links on the pages are external.com, and I want to switch 'em so I can navigate the site w/o going through the web server.

Anonymous said...

when using the proxy the browser freeze at "wating for..."

righteous said...

Make sure your servlet is deployed to the root context.

Mai said...
This comment has been removed by the author.
Anonymous said...

then what exactly this servlet do is to take the request and clone it and creat the connection?? or I miss understood something?

Anonymous said...

how we make these code on android ? your help is very usefull, please

Anonymous said...

Does this proxy work for multi part streams too ?

I have seen examples where they parse the request to file system and recreate the multipart request.

This just seems to pipe it. Please do let me know and thanks for your example.

righteous said...

This is not really a proxy. It simulates one. Multipart messages with files etc would not work.

Anonymous said...

Hi. Could someone please explain why buffers are only used for "POST"?

if(methodName.equals("POST")){
BufferedInputStream clientToProxyBuf = new BufferedInputStream(request.getInputStream());
BufferedOutputStream proxyToWebBuf = new BufferedOutputStream(con.getOutputStream());

while ((oneByte = clientToProxyBuf.read()) != -1)
proxyToWebBuf.write(oneByte);

proxyToWebBuf.flush();
proxyToWebBuf.close();
clientToProxyBuf.close();
}

And if it is "GET" how is the request data sent from proxy to web?

Thank you.

Anonymous said...

How do I run this program in eclipse? It is really complicated to run in eclipse.

Anonymous said...

how do I deploy the code? It does not seem to work in glassfish or eclipse. How will I know whether the website are going through our proxy or not? Where to change the proxy settings for the "Testing the Filters" section? At the browser or in glassfish.

Naviya Nair said...

Very interesting and good Explanation
ASP NET Training
ASP NET Training
ASP NET Online Training
C-Sharp Training
Dot Net Training in Chennai
Online .Net Training


MVC Training
WCF Training
Web-API Training
LINQ Training
Entity Framework
Training

Dot Net Interview Questions