Graph Advantage: Research Organizations

Many enterprises today build their business around research that involves piecing together meaningful data from the public domain for their customers. When trying to connect data across a domain in a meaningful way building around a graph database is a great tool because it models very well exactly how the business analysts at these research organizations are piecing together the real-world data they are finding during their research.

A business analyst may begin with one person and from there, move to the company they’re working for and then shifting to colleagues before moving on to places where their current colleagues previously worked, before finally settling for their past colleagues. Suddenly the business analysts has nearly finished building out an intuitive network of complex connections around this person of interest which would have been challenging and time consuming to try to represent in Excel.

Graph Database in Research Organizations

Research organizations are more than just managing large data volumes, their core goal is finding understanding that comes through research to gain insight of the available data. To properly leverage data relationships, a research organization requires a database technology that houses data relationship as a first-class entity.

As a native graph database, Neo4j provides several essential advantages for businesses today:

• Neo4j structures data connections precisely as they exist in the world around us with the contextually specific connections between entities as primary entities that can be explored in constant time.
• Real-time results for queries that are exploring the many different and complex connections Read More...... 

Graph Advantage: Connected Enterprise

The connected enterprise is the new norm. Traditional chain paradigm with sequential and siloed operations lacking a connection between customer and factory is no longer cutting it. Today enterprises are excepted to be sufficiently in touch and aware of how to interact with each uniquely individual person they are fortunate to call their customer. Technologies and operational procedures are rapidly changing to enable information to be connected and taken together to drive decision

making, direction, and interaction with the customer.

Connected Enterprise: Data Essentials

Connected data is the lifeblood of today’s enterprise. Yet, it’s frequently isolated in varying silos across an organization, with different accessibility, redundancy, quality, and varying data formats. Managing connected data involves identifying, cleaning, storing, and governing increased data volumes within an enterprise. Connected data involves essential information such as customers, users, products, services, sites, and business units.

Adequate practices for connected data management differ along a wide range of approaches. On one end, many believe that connected data should be united in one location; while on the other end, some recommend managing data assets from one application or service, even if information is housed in multiple locations.

In both cases, data architects require a data model that’s versatile and fluid when exceptions arise and business needs change. And the only model that can answer this is the graph database.

Data Management and Graph Databases

Enterprises today are flooded with “big data”, a majority of which is master data. Dealing with complex relationships between data points could be the biggest problem facing today’s

Read More......

Every Organization Needs a Knowledge Graph

A knowledge graph as it relates to individual organizations is a unification of information across that organization enriched with contextual and semantic relevance. Introducing a knowledge graph creates a comprehensive and baseline set of knowledge accessible by personnel, applications and customers alike to gain understanding and drive actions and direction.

This foundational knowledge graph is not only useful for people and applications, but provides a relevant and evolving dataset for sophisticated learning and intelligence software systems to utilize in providing personalized internal guidance as well as highly engaging interactions with customers.

Knowledge Sharing Falling Short

To engage all personnel in collaboration and knowledge sharing, a majority of organizations today have adopted social networking trends and offering different kinds of internal tools. However, such applications can generate large volumes of unstructured organization data stored in isolated systems across an organization. This attempt at creating a holistic understanding falls short because all this knowledge sharing and information isn’t actually being connected together.

The main result from this approach is a complex infrastructure containing data silos filled with duplicated, expired, and redundant information. This makes it hard to see the right information and acquire important insights. Organizations today need a graph data platform to support increasingly complex 

Read More......

systemd Neo4j service on Ubuntu

systemd ships as the default process manager on Ubuntu 15; get the neo4j systemd service file to install and learn about the configuration. systemd has a lot of great features and I encourage you to check it out as it’s a big improvement IMO from SystemV and Upstart. If you’ve started working with

systemd to manage your neo4j process, then you

might have stumble through the new setup. Here are a couple things I found helpful and some config that might get you moving along faster. I won’t go into detail on how to install services with systemd but it basically breaks down into these steps:

1.  Save service file:   /lib/systemd/system/neo4j.services 
2.  Reload config:  sudo systemctl daemon-reload 
3.  Enable on startup:  sudo systemctl enable neo4j.service 

Service file for systemd and neo4j

For those simply looking for a working neo4j.service file one is shown below. The rest of the writeup goes into detail how to install this service file and why some of the properties are set the way they are. This configuration assumes you have linked the neo4j script into /etc/init.d/neo4j and that your neo4j installation is at /opt/neo4j e.g. NEO4J_HOME=/opt/neo4j. The file below should be located at:

Read More......

Pairing Neo4j ElasticSearch: The Basics

There are a number of ways of integrating Neo4j with ElasticSearch. One common way was through the use of the Rivers plugin, but that was deprecated in ElasticSearch 1.5 and will likely be fully removed shortly after ElasticSearch 2.0. Going forward any integration will require a more sophisticated integration to index the desired nodes and relationships from Neo4j to ElasticSearch.

For those that don’t know, ElasticSearch is an open source search server based on Lucene that provides a distributed full-text search engine that utilizes JSON documents with a RESTful API.

Benefits of Neo4j ElasticSearch Pairing

ElasticSearch provides language analyzers, aggregations and other features right of the box, which are some of the reason it’s an ideal search solution to pair with Neo4j as opposed to trying to recreate all the text search capability within Neo4j. Some of the key advantages in the Neo4j ElasticSearch pairing include:

• Swift search against large data volumes
  Large and complex graph traversal queries spanning tens to hundreds of thousands of nodes that would take many seconds will take milliseconds with ElasticSearch because the query result is stored in a single document that can be easily indexed. The design of ElasticSearch is leaner and lot simpler compared to a database consisting of columns, rows, tables, fields, and schemas, which enables many documents with concise results to be indexed in a caching mechanism when the attribute nature of the query variations doesn’t explode the combinations needing stored.
• Document indexing to repository
  ElasticSearch can easily convert raw data (message files or log files) into internal documents. It then stores them within a basic data structure. Flowing documents to ElasticSearch is reliable to automate in a push fashion from Neo4j.
• Quick data access via de-normalized storage
  ElasticSearch will usually house a document for every repository in which it lives in. Full text searches are swift since documents are housed nearby to corresponding metadata within the index. The aggregators and language analyzers can then be used effectively to build together search queries that go from

Read More......

Neo4j Enterprise Cluster Basics

Neo4j Enterprise enables a high availability cluster using the PAXOSprotocol for cluster communication prior to 3.x and the RAFT protocol with the core-edge clustering model is now available in the current milestone releases. If you’re interested into diving deeper into specifications and the implementation of the new RAFT protocol, I suggest you checkout Jim Webber’s great overview in his keynote from GraphConnect SF 2015. One very useful feature coming in 3.x is the ability to read your own writes. Meaning you can require that the transaction with write you made to core is available on the edge server handling the read request before it returns your request.

So while that is coming in 3.x, what is the current landscape in 2.x?

Neo4j Enterprise Write Operations

When operating a Neo4j Enterprise cluster, there will always be one master instance and some number of slaves. Neo4j is capable of handling write requests on all instances, but that requires the slave to proxy the request to the master so it is best to separate reads and writes to ensure the master is the only Neo4j instance handling write requests.
Writes to the Neo4j master instance will be optimistically pushed to zero or more slaves as configured. This means the master will try pushing the successfully written transaction to the specified number of slaves prior to the write request completion. If the replication ends up failing for any reason, the transaction on the master will still remain successful although it will be different from the typical normal replication factor. The Neo4j slave instances will continue to pull for their updates at the configured interval so the writes will still eventually replicate and be available for read requests.

Neo4j Enterprise Master Re-Election

Whenever a Neo4j Enterprise graph database instance becomes

Read More......

Neo4j Production Ready: Deployment Basics

If you intend to perform a Neo4j production deployment successfully, you’ll likely think about the best application architecture to use and how you’ll operate your Neo4j Enterprise deployment at a scale. Some things you’ll need to think about should include how you intend to guarantee availability uptime, handle failures and efficiently facilitate zero downtime upgrades, which is really just the required baseline to be considered production ready. It may go without say, but to go to production without using Neo4j Enterprise is a huge risk to your applications availability.

Neo4j Deployment Options

In terms of deployment options, there are two ways in which you can incorporate the graph database Neo4j Enterprise version within your app. These can be:

• Using Neo4j embedded: This means you’ll be utilizing the Neo4j Java libraries and packaging it with the rest of your application code into a WAR or JAR file that is deployed to the Java server of your choice such as JBoss or Tomcat.
• Using Neo4j server: This means you’ll be utilizing the default Jetty server wrapper that is provided with Neo4j and communicating with the database over rest, which is the recommended approach for almost all applications because it keeps your database decoupled from your application and enables the two to be upgraded independently. In this architecture if you do need low-level access to the Java APIs you’ll be able to utilize unmanaged extensions that deploy directly to the Neo4j database.

Baseline for Deployment to be Production Ready

What are the minimum requirements needed to ensure your Neo4j Enterprise deployment is production ready?

• High Availability Clustering
  Thanks to advancements made in SaaS and mobile, enterprises have evolved to effectively engage with their customers over the years. This connected architecture offers

Read More......