FEP7: CQRS pattern on WebSphere Commerce

On February 28th, IBM made available WebSphere Commerce Feature Pack 7 (FEP7) for version 7.

Several new features are now available but there is one really interesting that introduce exciting architectural improvements.

In particular there is an architectural option who opens new scenarios for the ecommerce implementations based on Websphere Commerce and it reduces, very interesting from the business point of view, the licence investment needs.

With FEP7, Websphere Commerce becomes a platform able to implement CQRS pattern (“Command Query Responsibility
Segregation”) allowing to separate transactional traffic (“command model”) from browsing traffic (“query model”).

In general CQRS pattern is a natural evolution of CRUD model (“Create/Read/Update/Delete”) and allows ecommerce users to browse transparently two different infrastructures: one dedicated to the catalog navigation where no changes are needed on the underlying database and another specifically focused on managing transactions.

On WebSphere Commerce these two models are sharing the same database but the “query model” is based on REST services pointing to SOLR indexes.

The chance to (finally) split up the monolithic WebSphere Commerce architecture in two separated infrastructures introduced by FEP7 (image from IBM Knowledge Center)

FEP7 splitted infrastructure for Commerce and Search

allows to scale up independently the Search infrastructure (“query model”) from the Commerce one (“command model”).

Search infrastructure handle all browsing traffic without updating database model (navigation based on SOLR) and Commerce infrastructure manage all ecommerce related transactions (shopping carts, orders, users…)

Commerce and Search infrastructures can be separated on different Websphere Application Server clusters (image from IBM Knowlegde Center)

Commerce and Search separated clusters

and Search cluster can be reached from ecommerce frontend using his dedicated REST interface (lighter than the previous one) (image from IBM Knowlegde Center)

FEP7 against previous REST implementations

As the frontend can call Search infrastructure directly, the Commerce infrastucture has to manage transactional requests only.

And the consequence is you can size Search and Commerce infrastructure independently and also manage separated deployment processes getting more flexibility to adjust hardware requirements to your real needs. In retail customers for example you can improve your Search infrastructure to handle more browsing users during seasonal peaks without changing the transactional infrastructure depending on your user traffic pattern.

As shown on the pictures I got from IBM Knowledge Center it’s possible to create two different clusters, one for transactional and the other dedicated to no transactional traffic and using CBR (Content Based Routing) functions of a dispatcher to distribute REST calls to the different servers depending on their content (or better on his URL pattern).

Which are the advantages to distribute no transactional requests to a specific cluster?

  • Scale up independently resources dedicated to browsing and business transactions. If you need to manage traffic peaks probably your Search infrastructure will need more resources but your Commerce cluster will not. And it’s cheaper to add resources to a Web+SOLR 4.3 infrastructure than scale up DB2 or Oracle.
  • Better problem isolation. Separating the Commerce monolithic architecture in two infrastructures, issues on one environment are not affecting the other.
  • Higher ROI. Starting from now Websphere Commerce customers can use in his no transactional infrastructure (Search servers) the same number of cores (or PVUs) paid for the transactional infrastructure. It’s a simplification of course but we can say that splitting the ecommerce infrastructure using CQRS pattern allows you to “double” your Websphere Commerce investment in terms of licence.

To summary thanks to this new exciting features introduced by FEP7 that allows to separate transactional and no transactional traffic, it’s possible to separate scale up and optimize your investment on Websphere Commerce.

The spanish version of this article is available on atSistemas website.

Advertisements
Tagged with: , ,
Posted in search, websphere commerce

Move Websphere Commerce cache instances to eXtreme Scale: baseCache configuration

As explained in the previous post about WXS, we have an eXtreme Scale + Websphere Commerce infrastructure ready to use for functional tests.

In this post we move a step forward following the guidelines proposed in another developerWorks article

Configure WebSphere Commerce with WebSphere eXtreme Scale to improve performance, scale, and your competitive edge

To follow the general guidelines about WXS and WebSphere Commerce integration, I modified previous topology where catalog server were executed in a WebSphere application server JVM and I moved catalog server in a stand alone JVM.

This was quite easy: the installation steps are the same but when you configure the catalog server domain in the WAS console

Remote Catalog Server Domain

you have to choose the “Remote server” option

Remote Catalog Server Started in WAS Console

 

To startup the catalog server and the container you have to use the same grid configuration files used in the previous post

dynacache-objectgrid.xml

dynacache-deployment.xml

But this time you have to run the catalog server process manually first

startXsServer.bat catalogServer -serverProps C:\IBM\WebSphere\WXS\ObjectGrid\webspherecommerce\properties\objectGridServer.properties

using the same properties file we used in the setup part

objectGridServer.properties

and executing the container

startXsServer.bat wccontainer1 -objectGridFile C:\IBM\WebSphere\eXtremeScale\ObjectGrid\webspherecommerce\etc\dynacache-objectgrid.xml -deploymentPolicyFile C:\IBM\WebSphere\eXtremeScale\ObjectGrid\webspherecommerce\etc\dynacache-deployment.xml -catalogServiceEndPoints localhost:4809 -listenerPort 6601

After augmenting the profile, to switch the baseCache to WXS is quite simple: you only need to change the provider on “Dynamic cache service”

WXS provider in baseCache

Using a very simple cachespec.xml to cache Aurora Home page only, we can check if the WXS provider is working as expected

Loading Aurora home page, a new entry appears on cache monitor

Cache entries Home page Aurora cache monitor

How can we check if those entries were correctly stored in the grid?

We can do a first check using cache monitor accessing to the WXS Mbeans

Cachemonitor access to the WXS mbeans

But to get a full view about grid usage, we have to use WXS web console.

We already created all the references (catalog server and catalog service domain) in the previous post and we only have to connect using a browser

First of all looking at the cache contents (be careful only first 500 results will be returned)

WXS web console cache contents

And then the grid usage for example

WXS web console grid usage

A general suggestion to close: upgrade your WXS environment to 8.6.0.4 using FixCentral and open a PMR asking for the last cumulative fix available before starting a WXS implementation in a production environment.

 

Tagged with: , , ,
Posted in cache, websphere application server, websphere commerce, websphere eXtreme scale

Investigate intermittent production issues in WebSphere Application Server

In every production environment there is a very complicated kind of issues to solve: the intermittent ones.

The kind of errors that happens “sometime” only: tricky to catch and reproduce.

But from last August, there is a tool that can help us to investigate errors like these

[WebSphere Application Server Log Watcher: Using TrapIt.ear to watch for WebSphere Application Server events]

Executing from command line a very simple EAR file (trapit.ear) using Websphere Application client, you can define trigger events based on systemout messages.

Using a specific syntax you can generate javacores or activate traces or executing OS scripts depending on the messages are appearing on Websphere Application Server systemout file.

For example you can

sh> ./launchClient.sh /full/path/to/TrapIt.ear localhost:2809 EVENT=1,WSVR0605W JAVACORE=3,60

generate three javacores at 60 second intervals when the warning WSVR0605W shows up in the log.

Remember there will be always a small delay between the event and the action but this is a very interesting debug option for this kind of issues.

Tagged with: , ,
Posted in troubleshooting, websphere application server

WebSphere Commerce FEP7: new search infrastructure

The revolution is started!

FEP7 is finally available for customers through his IBM passport and a lot of architectural improvements will change Websphere Commerce implementations in the next months.

A new “Search” EAR is deployed in Websphere Application Server

Search EAR in WAS

and available in Websphere Commerce Developer

Search EAR in WCD

This will give to architects the chance to separate no transactional from transactional traffic in different WAS infrastructures

[Image from WebSphere Commerce v7 Infocenter: Deploying the WebSphere Commerce search server: Advanced configuration overview]

Are you ready for change? (big change)

Tagged with: , ,
Posted in search, websphere commerce

Move Websphere Commerce cache instances to eXtreme Scale: setup

To start playing with Websphere eXtreme Scale and Websphere Commerce as suggested on developerWorks, you could “improve” your Websphere Commerce Developer environment installing WXS and configuring a local grid.

The goal is to apply the developerWorks tips to Websphere Commerce cache instances (baseCache and data cache) and check WXS functionalities.

I started from a Websphere Commerce Developer v7 environment with FixPack 6 and FEP5 installed. Websphere Application Server is a WAS 7.0.0.19 Base.

To setup WXS environment you need two fundamental pieces: a catalog server and a container to host the grid.

WXS 8.6 allows many different topologies: in this case we will setup the catalog server in WAS to use PMI infrastructure and XIO. The container will be created as stand alone JVM because Websphere Commerce Test Environment is still on 32 bit only…

The high level steps to integrate WXS with Websphere Commerce are quite simple:

1) Install WebSphere eXtreme Scale as a standalone product (to execute WXS container locally)

2) Install the WebSphere eXtreme Scale on the WebSphere Commerce server (to execute catalog server in WAS)

3) Augment all WebSphere Commerce nodes with WebSphere eXtreme Scale using the WebSphere Application Server profile management tool.

Let’s see how to apply these steps in a Websphere Commerce Developer environment.

First of all, download WXS installation package with your Passport account (or through Partnerworld): the part number is CIK3HML and the file is “WS_XSCALE_V8.6.0.2_MP_ML.zip”

Add the WXS repository to your Installation Manager installation and install “WebSphere eXtreme Scale in a stand-alone environment” (step 1)

IM installation for WXS

Step 2 (“Install the WebSphere eXtreme Scale on the WebSphere Commerce deployment”) needs a couple of tricks to be done in a RAD + WAS Test environment environment.

In general we have to follow the instructions to install WXS client in a Websphere Application Server.

“If you want to install WebSphere eXtreme Scale or WebSphere eXtreme Scale Client within a WebSphere Application Server Version 7 environment, then complete the following steps:”

but in our case Websphere Application Server is already installed: it’s the WAS 7 Test Environment in RAD.

To be able to install WXS over WAS Test environment you have to

a) Create a local repository that contains the latest data downloading

http://public.dhe.ibm.com/software/websphere/downloads/was/import/was.7.0.import.zip

and adding it as a repository

b) Import WebSphere Application Server Version 7 into Installation Manager

Import WAS Test Environment in IM

pointing to the RAD Test Environment runtime for WAS 7 (“C:\WebSphere\IBM\SDP\runtimes\base_v7” in my case)

WAS 7 Test Environment imported in IM

Once imported

c) we can start to install WXS on the existing WAS Test Environment

WXS Stand alone installed

because WAS 7.0.0.19 Test Environment is now a known package group

WAS Test Environment Package Group in IM

Remember to not create a new profile at the end of the installation: we want to use the existing one.

Now step 2 is complete and we can go forward augmenting the existing WAS profile (step 3).

To do this, you first need to know WAS profile name

C:\WebSphere\IBM\SDP\runtimes\base_v7\bin>manageprofiles.bat -listProfiles
[WCTOOL~1_113452]

which normally starts with “WCTOOL” in Websphere Commerce Developer. In my case was “WCTOOL~1_113452”

Then augmenting the profile is quite simple

manageprofiles.bat -augment -profileName WCTOOL~1_113452 -templatePath C:\WebSphere\IBM\SDP\runtimes\base_v7\profileTemplates\xs_augment\default

The result is to get WXS administration menus in the WAS Console

WAS console WXS augmented

Now the infrastructural part is complete and we need to start up both catalog server and container to get a fully functional WXS environment.

First of all we need to create a catalog server domain and add a catalog server using WAS console. One catalog server and one container is enough to have functional tests with Websphere Commerce and WXS.

WAS console create catalog server and domain

The catalog server will be initially stopped because catalog server properties aren’t configured yet.

Using the available sample in the WXS installation (C:\IBM\WebSphere\eXtremeScale\ObjectGrid\properties\sampleServer.properties), I created a new specific properties file to configure it and I copied it in the WAS properties directory (C:\WebSphere\WCToolkit\wasprofile\properties)

The most important changes are:

listenerHost=<localHostname>

listenerPort=4809

publishHost=<localHostname>

transport=XIO

Restarting WAS after adding the catalog.services.cluster custom property to server1, the catalog server is started as expected (with XIO enabled)

WAS Console with Catalog Service started

If everything is fine, your sysout should report

[1/23/14 19:03:52:937 CET] 00000000 RuntimeInfo   I   CWOBJ0903I: The internal version of WebSphere eXtreme Scale is v7.0.2 (8.6.0.2) [cf21323.07120159].
[1/23/14 16:21:42:453 CET] 00000004 TCPChannel    A   TCPC0001I: TCP Channel XIOInboundTCP is listening on host *  (IPv4) port 4809.
[1/23/14 16:21:42:453 CET] 00000004 WSChannelFram A   CHFW0019I: The Transport Channel Service has started chain Chain-XIOInbound.
[1/23/14 16:21:42:468 CET] 00000004 XIORegistry   I   CWOBJ9054I: The eXtremeIO registry is using the endpoint ID [40000143bfb14544e0000498296fa73c].

Finally it’s time to startup the container.

As we are using XIO as transport and the catalog server is local, we need to change a couple of things using WXS cachespec sample files as templates (C:\IBM\WebSphere\eXtremeScale\ObjectGrid\dynacache\etc).

I created a new folder

C:\IBM\WebSphere\eXtremeScale\ObjectGrid\webspherecommerce\etc

and I copied there dynacache-objectgrid.xml and dynacache-deployment.xml changing a couple of properties.

Then we can start the container

startXsServer.bat wccontainer1 -objectGridFile C:\IBM\WebSphere\eXtremeScale\ObjectGrid\webspherecommerce\etc/dynacache-objectgrid.xml -deploymentPolicyFile C:\IBM\WebSphere\eXtremeScale\ObjectGrid\webspherecommerce\etc/dynacache-deployment.xml -catalogServiceEndPoints localhost:4809

To simplify container configuration I only changed dynacache-objectgrid.xml and dynacache-deployment.xml as suggested in the developerWorks article (“The configuration consists of two files that define both the structure of the grid and the deployment characteristics of the grid for dynacache:…”)

During the startup the container will connect with the catalog server

[1/24/14 8:34:04:999 CET] 00000001 ServerImpl    I   CWOBJ0204I: The transport type of this Server JVM is being determined by contacting the catalog service domain with the catalog service endpoints of: localhost:4809.
[1/24/14 8:34:06:233 CET] 00000001 ServerImpl    I   CWOBJ0200I: The transport type is eXtremeIO.
[1/24/14 8:34:35:624 CET] 00000001 ServerImpl    I   CWOBJ1001I: ObjectGrid Server wccontainer1 is ready to process requests.

The setup is now complete.

To check out how the grid is working, start the WXS web console and create the reference for a new catalog server

Create catalog server pointer in WXS web console

and then create a catalog service domain and add the existing catalog server

Catalog service domain in WXS web console

Finally you can check the grid: it’s up and running

WXS web console used entries

Thanks to Stefano Zauli to review the full procedure before publish.

Tagged with: , , ,
Posted in cache, was, websphere commerce

Precision Marketing basics: Customer Celebrates Birthday trigger

Usually when marketing team begin to take care about Websphere Commerce during a project, they start his precision marketing experience activating a couple of basics dialog activities: one using the “Customer Celebrates Birthday” trigger and the second is related to the “Customer abandon shopping cart” trigger.

Suppose you just need to send an email when a customer celebrate his birthday.

The dialog activity to do this is very simple

customerCelebratesBirthdayDialogActivity

but there are some requirements to get it work

1) Enable personalization ID
[Enabling personalization ID]

Modify wc-server.xml adding (or modifying) this tag

<PersonalizationId display=”false” enable=”true“/>

2) Enabling persistent sessions
[Enabling persistent sessions globally]

To activate the personalization ID you need to activate persistent sessions too (globally or at store level)
Modify wc-server.xml adding (or modifying) this tag

<PersistentSession cookieExpiry=”7″
delayNewPersistentGuestSession=”true” display=”false” enable=”true“/>

Remember to check dynacache implications when persistent sessions are enabled
[Dynamic caching considerations for persistent session]

3) User had to give his consent to receive emails

Quite simple to check

select * from emlusrrecv where users_id = XXXXX;

3) Configure outbound accounts to send emails at store level
[Configuring email activity accounts]

Remember to check “Time to start delivery”: it has to be 1 hour (or less) later than the “SendMarketingTriggers” scheduled job execution: I’ll explain why later.

Now you can create the dialog activity using “Customer Celebrates Birthday” trigger

[Trigger: Customer Celebrates Birthday]

The “Send Message” action needs a email template you have to create first using the standard editor o defining a custom one

[Sample JSP code: e-mail template]

If you choose the custom way (more flexible) you will probably need these two APARs

[JR43903: CMVC 223752 – TO MAKE CUSTOM EMAIL TEMPLATE JSPS CREATED AND REGISTERED IN EMLMSG SEARCHABLE VIA CMC]

[JR44839: CMVC 225162 – PREVENT INSERTED CATEGORY LINK IN EMAIL TEMPLATE IS NULL]

Creating the dialog activity you can check the option “Send Immediately”. What does it means?

If you choose to send immediately all the emails associated to this dialog activity, Websphere Commerce will create and send the emails right after the next “SendMarketingTriggers” execution

This is a good option for testing and debugging but in production it’s better to execute that task when the traffic is lower if a lot of emails will be generated. The “SendMarketingTriggers” task is planned at 2:00 AM out-of-the-box and the best practice is to set the “Time to start delivery” (specified earlier in the outbound account) one hour later.

To verify how many emails will be sent, you just need a simple query against USERDEMO table

select count(*) from userdemo where dateofbirth = ‘DD-MMM-YY’

There’s only one step left: activate the dialog activity.

If you followed all the steps and leaved the default settings, at 2:00 AM the “SendMarketingTriggers” will prepare all the emails for the users who will celebrated his birthday in the next X days (trigger parameter) and at 3:00 (suggested time to set “Time to start delivery”) those emails will be sent.

You can change those OOB settings

[Changing the daily scheduled time for marketing triggers and lists]

In general you will find a lot of suggestions on how to test dialog activities in this infocenter page

[Tips for testing triggers and actions in Dialog activities]

Tagged with: , ,
Posted in precision marketing, websphere commerce

How to cache wcf:getData results in JSPs

In the Aurora front-end (from FEP5 then) there is a new interesting way to store wcf:getData results and reuse it through the navigation

Using a new (and undocumented) custom tag, wcf:usebean

<wcf:useBean var=”cachedCatalogEntryDetailsMap” classname=”java.util.HashMap” scope=”request”/>

it’s possible for example to cache in an HashMap all the informations related to a specific product.

How is it done?

First of all you have to call CatalogNavigationView service to get the product data

<wcf:getData type=”com.ibm.commerce.catalog.facade.datatypes.CatalogNavigationViewType” var=”catalogNavigationView”
expressionBuilder=”getCatalogEntryViewAllByID” varShowVerb=”showCatalogNavigationView” maxItems=”1″ recordSetStartNumber=”0″>

and store the resulting product in a variable, in this case “catalogNavigationView”

Then with another new (and again undocumented) custom tag

<c:set var=”key1″ value=”${productId}+getCatalogEntryViewAllByID”/>
<wcf:set target = “${cachedCatalogEntryDetailsMap}” key=”${key1}” value=”${catalogNavigationView.catalogEntryView[0]}”/>

put the whole product object in the previous declared HashMap.

To use that object then it’s quite simple

<c:set var=”catalogEntryDetails” value=”${cachedCatalogEntryDetailsMap[key1]}”/>

and for example access the product thumbnail path

catalogEntryDetails.metaData[‘ThumbnailPath’]

Tagged with: , , ,
Posted in frontend, websphere commerce
%d bloggers like this: