Thursday, August 22, 2013

Entity-Attribute-Value

In the DB schema of a previous project (which was a 10 year old legacy system), there was a table with a set of attributes listed as rows instead of columns as shown below.


When asked why, it was told that there was a client request to add additional parameters to the product entity, but adding a new column was not feasible since the table contained millions of records. Instead they opted for this solution as a workaround. Two new tables were added and the "Product_Config_Param_Def" contained the new parameter definitions and "Product_Config_Param_Value" contained the corresponding values. This was flexible and allowed adding many more parameters without having to add columns. And the plus point is when there are products that don't have the entire set of parameters, we don't need to populate DB with nulls.

This is what is known as "Entity-Attribute-Value" Modeling (EAV) and it's a great way for us to get closer to the benefits offered by No SQL DBS with our relational DBs. No SQL DB of course offer us unlimited flexibility in adding dynamic attributes and is the way to go if our entities have large amount of varying attributes (In other words, when entities have different sets of attributes) and the relationships between entities is rather limited (Think FB accounts).

With EAV, data is modeled as attribute-value pairs.

EAV should be used with caution. Many identify it as an anti-pattern. It can be such a performance hit. If I may copy the example mentioned in http://tonyandrews.blogspot.com/2004/10/otlt-and-eav-two-big-design-mistakes.html

With no EAV:

select ename from emp where job=’CLERK’ and sal < 2000;

With EAV:

select ev1.name
from emp_values ev1, emp_values ev2 emp_values ev3
where ev1.code = ‘NAME’
and ev1.empno = ev2.empno
and ev2.code = ‘JOB’
and ev2.value = ‘CLERK’
and ev1.empno = ev3.empno
and ev3.code = ‘SAL’
and TO_NUMBER(ev3.value) < 2000;

Sunday, August 4, 2013

Camel-CXF: Java-First Service Implementation

Summarized from the documentation.

Web service interface is defined using a Service Endpoint Interface (SEI) and the WSDL is generated from it.

Steps
1. Implement & annotate the SEI.
2. Implement other requisite Java classes.
        - Any data types referenced by the SEI.
        - The implementation of the SEI. 3. Instantiate the Web service endpoint.
4. Generate the WSDL

1. Implement & annotate the SEI.

SEI is used as the;
• Base type of the Web service implementation (server side)—you define the Web service by implementing the SEI.
• Proxy type (client side)—on the client side, you use the SEI to invoke operations on the client proxy object.
• Basis for generating the WSDL contract—in the Java-first approach, you generate the WSDL contract by converting the SEI to WSDL.

In order to use the standard JAX-WS frontend, the SEI must be annotated with the @WebService annotation. Annotations are also used to customize the mapping from Java to WSDL.

import javax.jws.WebParam;
import javax.jws.WebService;

@WebService(targetNamespace = "http://demo.fusesource.com/wsdl/CustomerService/", name = "CustomerService", serviceName = "CustomerService", portName = "SOAPOverHTTP")
public interface CustomerService {
 public com.fusesource.demo.customer.Customer lookupCustomer(
   @WebParam(name = "customerId", targetNamespace = "") java.lang.String customerId);

 public void updateCustomer(
   @WebParam(name = "cust", targetNamespace = "") com.fusesource.demo.customer.Customer cust);

 public void getCustomerStatus(
   @WebParam(name = "customerId", targetNamespace = "") java.lang.String customerId,
   @WebParam(mode = WebParam.Mode.OUT, name = "status", targetNamespace = "") javax.xml.ws.Holder status,
   @WebParam(mode = WebParam.Mode.OUT, name = "statusMessage", targetNamespace = "") javax.xml.ws.Holder statusMessage);
}

2. Implement other requisite Java classes.

When you run the Java-to-WSDL compiler on the SEI, it converts not only the SEI, but also the classes referenced as parameters or return values. The parameter types must be convertible to XML, otherwise it would not be possible for WSDL operations to send or to receive those data types. In fact, when you run the Java-to-WSDL compiler, it is typically necessary to convert an entire tree of related classes to XML using the standard JAX-B encoding.

Each related class must have a default constructor .

public class Customer {
 protected String firstName;
 protected String lastName;
 protected String phoneNumber;
 protected String id;

 // Default constructor, required by JAX-WS
 public Customer() {
 }

 public Customer(String firstName, String lastName, String phoneNumber,
   String id) {
  super();
  this.firstName = firstName;
  this.lastName = lastName;
  this.phoneNumber = phoneNumber;
  this.id = id;
 }

 public String getFirstName() {
  return firstName;
 }

 public void setFirstName(String value) {
  this.firstName = value;
 }

 public String getLastName() {
  return lastName;
 }

 public void setLastName(String value) {
  this.lastName = value;
 }

 public String getPhoneNumber() {
  return phoneNumber;
 }

 public void setPhoneNumber(String value) {
  this.phoneNumber = value;
 }

 public String getId() {
  return id;
 }

 public void setId(String value) {
  this.id = value;
 }
}

3. Instantiate the Web service endpoint.

In Apache CXF, you create a WS endpoint by defining a jaxws:endpoint element in XML. The WS endpoint is effectively the runtime representation of the Web service: it opens an IP port to listen for SOAP/HTTP requests, is responsible for marshalling and unmarshalling messages (making use of the
generated Java stub code), and routes incoming requests to the relevant methods on the implementor class.

jaxws:endpoint => To integrate a WS endpoint with a Java implementation class.
cxf:cxfEndpoint => To integrate a WS endpoint with a Camel route.

Creating a Web service in Spring XML consists of the following two steps:
1. Create an instance of the implementor class, using the Spring bean element.
2. Create a WS endpoint, using the jaxws:endpoint element.


 
 
 
 
 
 
 

4. Generate the WSDL

To generate a WSDL contract from your SEI, you can use either the java2ws command-line utility or the cxf-java2ws-plugin Maven plug-in. Plug-in approached is given below.

Add plugin to the POM .


 ...
 
  2.4.2-fuse-00-05
 
 
  install
  
   ...
   
    org.apache.cxf
    cxf-java2ws-plugin
    ${cxf.version}
    
     
      org.apache.cxf
      cxf-rt-frontend-jaxws
      ${cxf.version}
     
     
      org.apache.cxf
      cxf-rt-frontend-simple
      ${cxf.version}
     
    
    
     
      process-classes
      process-classes
      
       org.fusesource.demo.camelcxf.ws.server.CustomerService
       
       
        ${basedir}/../src/main/resources/wsdl/CustomerService.wsdl
       
       true
       true
      
      
       java2ws
      
     
    
   
  
 
 

Thursday, July 25, 2013

REST API Design Rules

Found REST API Design Rulebook to be a useful guideline in designing REST APIs, and it's short n sweet with only 144 pages.



It has dished out a set of clear rules that are practical and easy to follow. Here is a summarized list of all the rules given in the book.

Identifier Design with URIs

URI Format
Rule: Forward slash separator (/) must be used to indicate a hierarchical relationship
Rule: A trailing forward slash (/) should not be included in URIs
Rule: Hyphens (-) should be used to improve the readability of URIs
Rule: Underscores (_) should not be used in URIs
Rule: Lowercase letters should be preferred in URI paths
Rule: File extensions should not be included in URIs

URI Authority Design
Rule: Consistent subdomain names should be used for your APIs
Rule: Consistent subdomain names should be used for your client developer portal

URI Path Design
Rule: A singular noun should be used for document names
Rule: A plural noun should be used for collection names
Rule: A plural noun should be used for store names
Rule: A verb or verb phrase should be used for controller names
Rule: Variable path segments may be substituted with identity-based values
Rule: CRUD function names should not be used in URIs

URI Query Design
Rule: The query component of a URI may be used to filter collections or stores
Rule: The query component of a URI should be used to paginate collection or store results

Interaction Design with HTTP

Request Methods
Rule: GET and POST must not be used to tunnel other request methods
Rule: GET must be used to retrieve a representation of a resource
Rule: HEAD should be used to retrieve response headers
Rule: PUT must be used to both insert and update a stored resource
Rule: PUT must be used to update mutable resources
Rule: POST must be used to create a new resource in a collection
Rule: POST must be used to execute controllers
Rule: DELETE must be used to remove a resource from its parent
Rule: OPTIONS should be used to retrieve metadata that describes a resource’s available interactions

Response Status Codes
Rule: 200 (“OK”) should be used to indicate nonspecific success
Rule: 200 (“OK”) must not be used to communicate errors in the response body
Rule: 201 (“Created”) must be used to indicate successful resource creation
Rule: 202 (“Accepted”) must be used to indicate successful start of an asynchronous action
Rule: 204 (“No Content”) should be used when the response body is intentionally empty
Rule: 301 (“Moved Permanently”) should be used to relocate resources
Rule: 302 (“Found”) should not be used
Rule: 303 (“See Other”) should be used to refer the client to a different URI
Rule: 304 (“Not Modified”) should be used to preserve bandwidth
Rule: 307 (“Temporary Redirect”) should be used to tell clients to resubmit the request to another URI
Rule: 400 (“Bad Request”) may be used to indicate nonspecific failure
Rule: 401 (“Unauthorized”) must be used when there is a problem with the client’s credentials
Rule: 403 (“Forbidden”) should be used to forbid access regardless of authorization state
Rule: 404 (“Not Found”) must be used when a client’s URI cannot be mapped to a resource
Rule: 405 (“Method Not Allowed”) must be used when the HTTP method is not supported
Rule: 406 (“Not Acceptable”) must be used when the requested media type cannot be served
Rule: 409 (“Conflict”) should be used to indicate a violation of resource state
Rule: 412 (“Precondition Failed”) should be used to support conditional operations
Rule: 415 (“Unsupported Media Type”) must be used when the media type of a request’s payload cannot be processed
Rule: 500 (“Internal Server Error”) should be used to indicate API malfunction

Metadata Design

HTTP Headers
Rule: Content-Type must be used
Rule: Content-Length should be used
Rule: Last-Modified should be used in responses
Rule: ETag should be used in responses
Rule: Stores must support conditional PUT requests
Rule: Location must be used to specify the URI of a newly created resource
Rule: Cache-Control, Expires, and Date response headers should be used to encourage caching
Rule: Cache-Control, Expires, and Pragma response headers may be used to discourage caching
Rule: Caching should be encouraged
Rule: Expiration caching headers should be used with 200 (“OK”) responses
Rule: Expiration caching headers may optionally be used with 3xx and 4xx responses
Rule: Custom HTTP headers must not be used to change the behavior of HTTP methods

Media Type Design
Rule: Application-specific media types should be used
Rule: Media type negotiation should be supported when multiple representations are available
Rule: Media type selection using a query parameter may be supported

Representation Design

Message Body Format
Rule: JSON should be supported for resource representation
Rule: JSON must be well-formed
Rule: XML and other formats may optionally be used for resource representation
Rule: Additional envelopes must not be created

Hypermedia Representation
Rule: A consistent form should be used to represent links
Rule: A consistent form should be used to represent link relations
Rule: A consistent form should be used to advertise links
Rule: A self link should be included in response message body representations
Rule: Minimize the number of advertised “entry point” API URIs
Rule: Links should be used to advertise a resource’s available actions in a state-sensitive manner

Media Type Representation
Rule: A consistent form should be used to represent media type formats
Rule: A consistent form should be used to represent media type schemas

Error Representation
Rule: A consistent form should be used to represent errors
Rule: A consistent form should be used to represent error responses
Rule: Consistent error types should be used for common error conditions

Client Concerns

Versioning
Rule: New URIs should be used to introduce new concepts
Rule: Schemas should be used to manage representational form versions
Rule: Entity tags should be used to manage representational state versions

Security
Rule: OAuth may be used to protect resources
Rule: API management solutions may be used to protect resources

Response Representation Composition
Rule: The query component of a URI should be used to support partial responses
Rule: The query component of a URI should be used to embed linked resources

JavaScript Clients
Rule: JSONP should be supported to provide multi-origin read access from JavaScript
Rule: CORS should be supported to provide multi-origin read/write access from JavaScript

Tuesday, July 16, 2013

Fuse ESB Demo Code

Fuse ESB Enterprise documentation is making use of demonstration code to illustrate the capabilities of the system. Explanations are not very clear to follow without the code, but unfortunately the code is for subscribers only.

Downloading the demonstration package

The source code for the demonstrations is packaged as a Zip file, cxf-webinars-assembly-1.1.1-src.zip, and is available from the following location:
http://repo.fusesource.com/nexus/content/repositories/subscriber/org/fusesource/sparks/fuse-
webinars/cxf-webinars-assembly/1.1.1

When you try to navigate to this location with your browser, you will be prompted for a username and password. Enter your subscription login credentials to gain access to this directory and click on cxf-webinars-assembly-1.1.1-src.zip to start downloading.

Did a little Googling to see if any generous developer has shared the code, and look what we found!

The actual git repository they use for storing these projects is not restricted.

http://fusesource.com/forge/git/sparks.git/?p=sparks.git;a=tree

It's not possible to clone it, since it requires authentication. But can click on the snapshot  (http://fusesource.com/forge/git/sparks.git/?p=sparks.git;a=snapshot;h=HEAD;sf=tgz) and the entire lot is downloaded.

Wednesday, July 10, 2013

Frequently Used Linux Commands

For my easy reference,

find . -name *.jar | xargs grep -i 'ABC.class'
# Find a jar file which has ABC.class in it

printenv
# Environment variables

printenv | grep proxy
# Environment variables with proxy in name

export https_proxy=http://x.x.x.x:3128
# Set environment variable

sudo su
# Login as super user

rm -rf folderToDelete
# Delete the folder and the contents

netstat -anp|grep :7001
# Find a process binding a port

kill -9 21286
# Kill process

telnet hostA 1521
# Telnet a port

nslookup hostA
# 

uname -a
# kernel-name, nodename, processor, OS etc

lsb_release -a
# OS version

cat /etc/*release 
# OS version

df -h
# file system sizes & free space

du -s folderloc
# folder size

du --si -s file 
# File size shown in G,M

ssh root@serverA
# Connect to serverA as root

scp root@serverA.abc.com:/path/a.log b.log
# Copy a file from serverA

scp -r /path/folderA  userB@serverB:/path/folderB
# Copy a folder to serverB

tar -cf archive.tar foo bar
# Create archive.tar from files foo and bar.

tar -xf archive.tar
# Extract all files from archive.tar.

vpnc-connect
# VPN connect

date
# Current date

date +%T -s "15:22:13"
# Update time

split --bytes=50m largeFile.log splitFile_
# Split a file to files with 50m


Common Options

-f, --force
-r, -R, --recursive
-a, --all
-v, --verbose
--help
--version