This chapter introduces the use of visualization widgets to display MarkLogic Server search results. These widgets provide a means to display data in picture or graph form. You can incorporate visualization widgets directly in the display output of applications you write, or incorporate them in applications created with Application Builder.
The following is the search page from the Oscars application created by Application Builder. Every object on the page is a visualization widget.
This chapter contains the following sections:
MarkLogic Server provides six visualization widget types:
This section contains the following subsections:
The quickest and easiest way to learn how to use visualization widgets is to use Application Builder to build an example Oscars application, as described in Building the Oscars Sample Application in the Application Builder Developer's Guide.
To add all of the widget types to the Oscars application, navigate to the Assemble page in Application Builder and do the following:
In general, clicking on a widget item (for example, a bar on a bar chart) changes the search query for the page and, as a result, the sidebar, results, and any other widgets on the page are updated to reflect the new search term. The only exceptions to this is the line chart widget and clicking on a marker in a map widget.
Searches initiated through the widgets are stemmed searches, which means the results returned for a term like 'name' also include ‘names,' ‘named', and so on.
The following Line Chart shows the search results on the 'decade' facet, with each dot indicating the number of nominations during that 'decade.' The chart's y-axis is the number of nominations, and its x-axis is the year, going from the first Oscars in 1920 to the present.
If you mouse over the Line Chart, a tool tip window appears, giving the date and number of nominations associated with the nearest data point, for example 'Decade: 1970s' and 'Nominations: 44'
Line charts are currently read-only and do not update the sidebar, results, or other widgets on the page.
The following Bar and Column Charts show the search results on the 'decade' facet.
If you mouse over a Bar or Column Chart, a tool tip window appears, giving the number of nominations (either by decade or film, depending on which chart) and film name or decade (again, depending on which chart) associated with the nearest bar. Clicking and dragging the mouse over the chart does not do anything. Clicking a single bar changes the Results area of the page to only show the search results associated with the film or decade, depending on the chart. Clicks on charts also change other widgets on the page, applying the value of the clicked bar as an additional search term to those widgets.
Note in the chart shown below, which has had the 1920s bar clicked, the number of Results is 10, the same as the number of results which occurred in the 1920s.
The following Pie Chart shows the search results on the 'name' facet.
If you mouse over a Pie Chart, a tool tip window appears, giving the name value and number of search matching nominations associated with the nearest section, for example 'Clint Eastwood: 3'. Clicking on a single Pie Chart section updates the Sidebar and Results sections, so that they only show the results for the Name value associated with that Pie Chart section. For example, if you click on the section for Clint Eastwood, only search results associated with Clint Eastwood appear in the page's Sidebar and Results sections.
Before you can select a map widget, you must create a geospatial element pair index. Navigate to the Search tab in Application Builder, click Add New at the bottom of the page, select Geospatial, and name it ‘map'. Click Create Geospatial Constraint.
Navigate to the Assemble page, select the map widget, and assign it the map facet. Deploy the application.
If you are writing an application in which you expect over 25,000 map loads per day, you will be required to purchase an a Map API Key from Google. Enter the key in the Map API Key field in the Map Setup section.
The map widget in the deployed application shows the search results on the entire Oscars data set. Each circle represents a cluster of markers. The color of and the number inside the marker cluster represents the number of results. If you select facets to narrow the data set, the map will change to represent that subset of results.
Clicking and dragging the mouse over the map widget causes the map display to pan or move in the drag direction. If you click on marker cluster, you can drill down to more marker clusters and finally to individual markers, which represent each nominee's birthplace.
Unlike the other widgets, clicking on a marker does not update the Sidebar or Results. Rather, it open an info window containing the actor's name, birth date, and birth place, such as 'Joan Crawford / Born 1905-03-23 / San Antonio, Texas'.
The Sidebar and Results will be updated if you use the Draw Shape tool to select one or more markers on the map. Click on the polygon icon to set the map to Draw Shape mode:
For example, to update the search results based on two markers, left-click points and drag lines around the markers to draw a polygon as shown below:
The front end architecture of the widgets is based on modules that can both send messages and receive them, but no module has any knowledge of other modules or the page at large. Each module has a listener that listens for messages that meet specific parameters. Similarly, a module sends out messages into the common medium without specific knowledge of the recipient of that message, but other modules with the proper listeners will receive those messages. In this fashion, module can be added and removed in realtime without disrupting the flow of the application, and there is no need for a controller to keep tabs on all of the application's modules. This maintains a strong separation of interests and allows for easier unit testing.
Modules are not aware of other modules. They can only trigger events and listen for events. The only information exchange between modules is through listening to messages in the common medium. There is no tracking of the particular source of information or particular destination. Messages are released into the common medium and read by those modules that have the proper listeners for picking up those particular messages.
The widget architectural model makes heavy use of event bubbling in the Document Object Model (DOM). There are two basic types of modules: Widget modules and controller modules. In most instances, there is a single controller module and one or more widget modules.
This section contains the following topics:
Using event handlers, widgets listen for newData events in order to listen for new pieces of data to render. Similarly, widgets can generate newQuery events in order to communicate a change in the query based on internal changes to the widget.
Two simple degenerate cases are a search bar widget and a results widget:
Widgets usually use a hybrid of the two types, both displaying information and updating the query in order to update the view into that query. An individual widget doesn't necessarily know the whole query, but tends to have control over a portion of it. Most visualizations take ownership of a particular facet and apply or remove a filter on the search based on that facet.
The controller is a module just like a widget. It's main job is listening to newQuery events and posting newData events. The controller makes use of a private comm (communicator) object that is responsible for making Ajax calls to the server for new data. For example, when panning a map widget, the map needs to update the marker clusters, which requires a query. The map widget sends out an event that the controller hears and passes on to the comm object, which requests the data and sends it back to the map widget.
The controller and comm object offer a lot of extensibility room for features like caching and data manipulations. The controller accepts an option configuration object that specifies things like the endpoint to access on the server, the application wrapper, the widget class, etc. By specifying a limited application wrapper, multiple controllers, for example, can be created on the page each running a separate application. They only respond to messages within their application container. Similarly, two applications can share a container on the page and use different widget classes to control their individual widgets.
The controller receives newQuery events and translates the query object into a structured query, which is appended to the Ajax call generated in the comm object.
Widgets are identified and linked to a controller using an HTML class. Applications generated by Application Builder name this class 'widget.'
For example, to add the results widget to the page, specify the following:
<div id="results" class="widget"></div>
Interactions within widgets consist of selecting data or pagination. Each widget has an internal updateQuery
method that can be called with a query object to send out a newQuery event. In the case of visual widgets, it makes "selections" on facets to update the query. A column chart or pie chart takes a single selection and a map takes an area (a polygon). For example, clicking on a bar in a column chart generates a selection event which bubbles up from the DOM and is read by the widget, which translates the selection into a newQuery event. The translation logic lives in the widget.
The basic idea of a shadow query is that, with many widgets on a page, multiple selections can be made at once and, if widgets are updating with each search, clicking on a widget essentially nullifies its further usefulness in searching because it will only display the one result that is the current selection.
The idea of shadow queries is to provide a 'context' or 'shadow' for these selected widgets. When a widget is selected, the selection is highlighted, but the other categories remain visible while the other widgets on the page update. When the next widget is selected, two queries are fired, one is the general search that's updating unselected widgets and the results pane; the other is the 'shadow query' for the selected widget. This shadow query is a copy of the main search query, except without the selected widget's contribution to that search. This renders a view "around" the selection in the widget and greatly expands the power of widgets on the page by offering a very deep view into the data.
Shadow Queries work by taking advantage of the datastream. The datastream is set in two places, once in each widget upon instantiation (datastream
is a parameter in the createWidget
method described in ML.createWidget (Null Widget) Method) and also upon creation of shadow queries in the controller. The datastream is used to match a shadow query to the widget that it is shadowing.
When a widget sets off a shadow query by making a selection, it will not render the next update it receives, as that update is informing the widget of its already current state.
Some code is common to both chart widgets (Bar, Column, Line, and Pie Charts) and map widgets (Point and Heat).
In your <head>
element, first, add your external library dependencies for line, bar, column, and pie charts:
<!-- external library dependencies --> <script src="lib/external/jquery-1.7.1.min.js" type="text/javascript"></script> <script src="lib/external/highscharts.src.js" type="text/javascript"></script>
If you are using map widgets, you must also include the following external libraries:
<!-- external web map files... --> <script src="http://maps.googleapis.com/maps/api/js?key=&sensor=false& region=US&libraries=drawing" type="text/javascript"> </script> <script src="lib/external/mxn-2.0.18/mxn.js?(googlev3)" type="text/javascript"></script> <script src="lib/external/heatmap.js" type="text/javascript"></script> <script src="lib/external/heatmap-gmaps.js" type="text/javascript"></script> <script src="lib/external/markerclusterer_min.js" type="text/javascript"></script>
Then add calls to the internal widget framework library. Of course, if you are not using histograph widgets, you do not need to include the chart.js
library, and if you are not using map widgets you do not need to include the map.js
library:
<!-- internal widget framework library --> <script src="lib/controller.js" type="text/javascript"></script> <script src="lib/widget.js" type="text/javascript"></script> <script src="/lib/viz/chart/chart.js" type-"text/javascript"></script> <script src="/lib/viz/map/map.js" type-"text/javascript"></script>
Finally, you may want to define a CSS style for your widget elements: This is optional, as there is a set of default CSS styles for the various widget types in the library (see /lib/viz/chart.css
and /lib/viz/map.css
). If you do define a custom style, do something similar to the following, defining your widgets' display parameters to meet your application's requirements. Of course, you do not have to call your element widget
, especially if you want to have different display parameters for different visualization widgets and thus need to define several different elements.
<style type="text/css"> .widget { width: 500px; display: block; float: left; } </style>
You must define each widget as a separate <div>
element with either the default class "widget"
or the class name you specified when initializing widgets with ML.controller.init
and its widgetClass
option
(see ML.controller Methods). In the example later on, which has three visualization widgets (two charts and one map), you have:
<div id="decadeContainer" class="widget"></div> <div id="actorContainer" class="widget"></div> <div id="locationContainer" class="widget"></div>
ML.controller.init({proxy: "proxy.php"});
For details on how to define the PHP proxy, see Set Up A Proxy
<script>
, define configuration variables for each of your visualization widgets. The configuration parameters differ for chart and map widgets (and also for Point Map and Heat Map map widgets), so see below for detailed examples and later for a complete list of possible configuration parameters for all visualization widget types. But placeholders for the three widgets in our full example would look like:var yearConfig = { ...chart configuration parameters... } var actorConfig = { ...chart configuration parameters... } var mapConfig = { ...point and heat map configuration parameters... }
<script>
. ML.controller.init
takes as its argument the variable you defined back in 1) that contains the information about the proxy server and this application's search endpoint, in this case config
. ML.chartWidget
and ML.mapWidget
both have the same signature of widget element name, widget type, and configuration parameters variable name, and only differ in whether a chart or map widget is created.
ML.controller.init(config); //connect to the var with our proxy value //chartWidget can take 'line' or 'bar' or 'column' or 'pie' //mapWidget just takes 'map' ML.chartWidget('decadeContainer', 'line', yearConfig); ML.chartWidget('actorContainer', 'bar', actorConfig); ML.mapWidget('locationContainer', 'map', mapConfig);
To initialize the widgets' display, use the use the ML.controller.loadDat
a method of the controller:
ML.controller.loadData();
This section describes how to add visualization widgets to an existing search application. In order to add visualization widgets, you must do the following:
It is assumed that you have a MarkLogic database set up when you begin the following process.
restaurants
, in MarkLogic Server.-modules
. For example, if you name the REST server restaurants
, a database, named restaurants-modules
, is automatically created. In this example, the REST server is located at: myhost:5437.restaurants
database. These correspond to the facets to visualize using the widget: scalar type | localname |
---|---|
string |
Type |
int |
SqFtEst |
restaurants
database:xquery version "1.0-ml"; xdmp:document-insert( "/1.xml", <restaurant> <id>1</id> <DBA> 123 Deli - Lee's </DBA> <Type> Restaurant, < 500 sq' </Type> <SqFtEst> 500 </SqFtEst> <StreetAddress> 123 1st St </StreetAddress> <City> San Francisco </City> <State> CA </State> <FullAddress> 123 1st St, San Francisco CA </FullAddress> <Latitude>37.78946</Latitude> <Longitude>-122.39717</Longitude> </restaurant>), xdmp:document-insert( "/2.xml", <restaurant> <id>2</id> <DBA> Dave's Dive </DBA> <Type> Restaurant, < 700 sq' </Type> <SqFtEst> 700 </SqFtEst> <StreetAddress>2 1 Mission St </StreetAddress> <City> Bismarck </City> <State> ND </State> <FullAddress> 21 Mission St, Bismarck ND </FullAddress> <Latitude>46.48</Latitude> <Longitude>-100.47</Longitude> </restaurant>), xdmp:document-insert( "/3.xml", <restaurant> <id>3</id> <DBA> Wayne's Steakhouse </DBA> <Type> Restaurant, < 1000 sq' </Type> <SqFtEst> 1000 </SqFtEst> <StreetAddress> 444 Feick St </StreetAddress> <City> Albany </City> <State> NY </State> <FullAddress>444 Feick St, Albany NY</FullAddress> <Latitude>42.40</Latitude> <Longitude>-73.45</Longitude> </restaurant>), xdmp:document-insert( "/4.xml", <restaurant> <id>4</id> <DBA> Megha </DBA> <Type> Restaurant, < 1000 sq' </Type> <SqFtEst> 1000 </SqFtEst> <StreetAddress> 4531 Front St </StreetAddress> <City> Carlsbad </City> <State> NM </State> <FullAddress> 4531 Front St, Carlsbad NM </FullAddress> <Latitude>32.26</Latitude> <Longitude>-104.15</Longitude> </restaurant>), xdmp:document-insert( "/5.xml", <restaurant> <id>5</id> <DBA> Gordon's Grubhaus </DBA> <Type> Restaurant, < 300 sq' </Type> <SqFtEst> 300 </SqFtEst> <StreetAddress> 3421 Capital Wy </StreetAddress> <City> El Paso </City> <State> TX </State> <FullAddress> 421 Capital Wy, El Paso TX </FullAddress> <Latitude>31.46</Latitude> <Longitude>-106.28</Longitude> </restaurant>)
restaurants-modules
database. This exposes the facets required for the visualizations. You must be logged into MarkLogic Server as a user with the rest-admin
privilege to run this query.
xdmp:http-put("http://myhost:5437/v1/config/query/all", <options xmlns="xdmp:http"> <authentication method="digest"> <username>admin</username> <password>admin</password> </authentication> <headers> <content-type>application/xml</content-type> <accept>application/xml</accept> </headers> </options>, text { xdmp:quote( <options xmlns="http://marklogic.com/appservices/search"> <constraint name="Type"> <range type="xs:string" facet="true"> <element ns="" name="Type"/> <facet-option>limit=10</facet-option> <facet-option>frequency-order</facet-option> <facet-option>descending</facet-option> </range> </constraint> <constraint name="SqFtEst"> <range type="xs:int" facet="true"> <element ns="" name="SqFtEst"/> <facet-option>limit=5</facet-option> <facet-option>frequency-order</facet-option> <facet-option>descending</facet-option> </range> </constraint> </options> ) } )
<server>:<rest-port>
identify your REST server. You should see an options node file with all your constraints set up.http://<server>:<rest-port>/v1/config/query/all
/var/www/html
.http://localhost:8000/appbuilder/customappfiles.xqy
customappfiles.xqy
file and copy the five subfolders (constraint, css, images, lib, and skins) to the public directory for your PHP-enabled Apache server. In this example, there is no ‘application' directory, as there is in applications created by Application Builder.
proxy.php
, in the public directory for your PHP-enabled Apache server with the following contents. Make sure the CURLOPT_URL
option points to your restaurants
server (myhost:5437
, in this example) and that the CURLOPT_USERPWD
option contains your login credentials for MarkLogic Server.<?PHP $queryString = $_SERVER['QUERY_STRING']; parse_str($queryString, $vars); unset($vars['proxyPath']); $queryString = http_build_query($vars); $options = array( CURLOPT_URL => 'http://myhost:5437' . $_GET["proxyPath"] . '?' . $queryString, // location of the rest server. Be sure to preserve the query string unaltered CURLOPT_HTTPAUTH => CURLAUTH_DIGEST, // it is very important to set the authentication type to digest CURLOPT_USERPWD => 'admin:admin', // this is your username and password for the server. be sure your role has read permissions on the database CURLOPT_HTTPHEADER => array('Content-type: application/json'), // the server won't respond properly unless the content-type is application/json ); // Only set POST options for POSTs, not GETs if ($_SERVER['REQUEST_METHOD'] === 'POST') { $options[CURLOPT_POST] = 1; // queries are sent in the post body, with options like pagination in the query string $options[CURLOPT_POSTFIELDS] = file_get_contents('php://input'); // we are not sending a traditional form map through the post, so avoid using the standard $_POST } $ch = curl_init(); // create curl curl_setopt_array($ch, $options); // set curl options curl_exec($ch); // execute curl ?>
<html> <head> <title>Custom App</title> <!-- external library dependencies --> <script src="lib/external/jquery-1.7.1.min.js" type="text/javascript"></script> <script src="lib/external/highcharts.src.js" type="text/javascript"></script> <!-- internal widget framework library --> <script src="lib/controller.js" type="text/javascript"></script> <script src="lib/widget.js" type="text/javascript"></script> <!-- visualization framework libraries --> <script src="lib/viz/chart/chart.js" type="text/javascript"> </script> <link media="screen, print" href="lib/viz/chart/chart.css" rel="stylesheet" type="text/css"> <style type="text/css"> .widget { width: 500px; display: block; float: left; } </style> </head> <body> <h2>MarkLogic Custom Application</h2> <div id="typeContainer" class="widget"></div> <div id="sqftContainer" class="widget"></div> <script type="text/javascript"> <!-- If you have a proxy, identify it here. --> var config = { proxy: "proxy.php" }; var typeConfig = { constraint: 'Type', constraintType: 'range-unbucketed', dataType: 'xs:string', title: 'Type', dataLabel: 'Type' }; var sqftConfig = { constraint: 'SqFtEst', constraintType: 'range-unbucketed', dataType: 'xs:int', title: 'Square Feet', dataLabel: 'Square Feet' }; ML.controller.init(config); ML.chartWidget('typeContainer', 'column', typeConfig); ML.chartWidget('sqftContainer', 'bar', sqftConfig); <!-- This will trigger the intial empty search --> ML.controller.loadData(); </script> </body> </html>
http://myhost/index.html
There are several limitations common to most or all widget types. These are:
In order to run a custom app, you generally need to run your queries through a proxy in order to authenticate on your MarkLogic REST server instance.
There is a proxy variable in the controller configuration that should be set to the base path of your proxy. The controller will append a 'proxyPath' URL parameter that your proxy should read. The proxyPath will point to the absolute path on the MarkLogic REST sever instance that the query needs to be proxied to. If the proxy is called with a POST, it should pass along the POST data to the MarkLogic REST server instance unchanged. It should also pass all url parameters aside from the proxyPath parameter. In the case of a GET request, just the url parameters are needed. The responses should be passed straight through unaltered.
An example PHP implementation is described in Example: Adding Widgets to Applications. Note that several options, such as the location of the REST server and the username and password values, may not be the same as your values. Edit the code so that it uses your values. Put this code in a publicly accessible file on your PHP server, for example, proxy.php
, which will serve as your application's search endpoint.
After confirming you have the proxy working correctly, add a proxy
option to the ML.controller.init
method in your index.html
page to identify the PHP proxy. For example:
ML.controller.init({proxy: "proxy.php"});
This section serves as a reference to the configuration options for each visualization widget and related methods. Bar, Column, Line, and Pie Charts all have the same configuration options. While it may seem odd to include Pie Charts with the others, if you consider a "slice" of a Pie Chart as serving the same basic purpose as a bar/column/datapoint on the other charts, it should make sense.
The topics in this section are:
The ML.controller.init
method has the signature:
ML.controller.init ({option1: value1, option2: value2...});
The 'null widget' is the core class of a widget in the application. The role of the null widget is to handle general communication behavior and incoming data. Visualizations, such as charts and maps, as well as controls, such as the search bar or displays like the results pane, are all extensions of the basic null widget.
ML.createWidget(container, renderCB, datastream, constraintType)
The method used to create line, bar, column, and pie widgets is:
ML.chartWidget(containerID, type, config);
The ML.chartWidget
configuration options are:
The method used to create map widgets is:
ML.mapWidget('locationContainer', 'map', mapConfig);
The following are configuration options for map widgets. They are used to define a configuration variable similar to:
var mapConfig = {
constraintType: 'geo',
dataStream: 'results',
statusContainerId: 'debug',
width: 698,
height: 512,
...
};
The point map configuration options are:
In addition to the point map options above, heat maps also have the following options:
Query Structure (can be empty)
The search results format (REST API)
Field | Description |
---|---|
value | 'name' of facet value that is selected and will be used to update the search. For example, in the actors facet of the Oscars app, this might be "Humphrey Bogart" |
constraintType | Must be "range", "constraint", or "geo". Determines how the structured query is generated. Map data is always of type geo. Other data can be range or constraint, depending on whether they are range indexes or collections on the server. |
geo | The geo property contains one of the following:
|