Skip to content

Neo4j Graph database learnings

March 25, 2015

Performance learnings:

Tuning of queries:

Relations:

Dynamic generation:

Nodejs creating large excel files

February 19, 2015
tags: ,

We wanted to created excel files that is having a few hundred thousand records – unfortunately the plugin msexcel builder did not work for us. other alternatives were to create a tabbed, or comma separated csv file and then open it in xls.

the msexcel builder plugin: had this very interesting concept of creating xls files using the concept of open office XML.

https://msdn.microsoft.com/en-us/library/office/gg278309.aspx

The only problem was that it was writing out all content at once, causing memory problem as data was big.  So we modified this to do appendFile instead of using writeFile. We also had to do away with the concept of sharedStrings, because when data is big this array itself would have costed us memory.

On testing with few thousand data, we also found traces of xml-builder objects lingering in memory – when we checked through ::findjsobjects in mdb(smartos) , So in the appending of content which was happening frequently we removed usage of xmlbuilder and stuck to plain old xml string formation. (Even though it looks less appealing it improved speed and reduced memory usage).

Next bottleneck that came was in usage of EasyZip, EasyZip – does full file read and writes out the binary content in the zip file which is actually the xlsx. So for this , the alternative we thought of was to use the OS specific zip command so that in memory usage reduced.

Angularjs directive scope sharing

October 29, 2014
tags:

Basics
directives are interpreted by compiler ($compile) service that attach events/change dom. $interpolate is used to evaluate the expressions. $compile invokes directive once using $injector.invoke.

Types
template (html content) or template-url linked to directive (template-url can also be fn(ele,attrs) – Cannot access scope inside template-url as scope created later.
You can create nested directives

Scope
transclude : Nested scope. Uses outer scope instead of inner scope of directive.
isolated scope – isolated scope – isolates all variables except those you mention.
. used to separate directive scope from parent except for specific scoped variables you want to pass from parent ctrl to directive you use scope inside directive , like:

scope: {
directiveScope: '=scopeAttr'
}

Return object from directive instead of fn.
Create prefixes for your own directives
Cleanup: element.on(‘$destroy’, …) or scope.$on(‘$destroy’, …) when directive is removed.
only use transclude: true when you want to create a directive that wraps arbitrary content.
=scope and &externalBehavior in outer context : to expose API

ElasticSearch special character search

October 29, 2014


GET index/type/_search
{
"size": 1000,
"fields": [
"fieldName"
], "query": {
"match": {
"_all": {
"query": "identifierspecialcharnext:*",
"type": "phrase_prefix"
}
}
}
}

Gorm Mongodb Connection timeout resolution

October 1, 2013

We faced connection timeout when using mongodb driver:

http://jira.grails.org/browse/GPMONGODB-307

At this time Mongodb has released their new mongodb java driver version  that is a fix for issue

https://jira.mongodb.org/browse/JAVA-853

To use this latest driver do the following, Set excludes in the plugin dependencies.And add this jar to dependency list:

dependency{
compile "org.mongodb:mongo-java-driver:2.11.3"
}
plugins{
compile (":mongodb:1.3.0"){
excludes "mongo-java-driver";
}

As of date today – 1st oct, 3.0.0 is not release so you will get below error if u add it :
Loading Grails 2.3
| Configuring classpath
| Error Resolve error obtaining dependencies: Could not find artifact org.mongodb:mongo-java-driver:jar:3.0.0 in grailsCentral (http://repo.grails.org/grails/plugins)

This should resolve below error:
Connection wait timeout after 120000 ms. Stacktrace follows:
com.mongodb.DBPortPool$ConnectionWaitTimeOut: Connection wait timeout after 120000 ms
at com.mongodb.DBPortPool.get(DBPortPool.java:222)
at com.mongodb.DBTCPConnector$MyPort.get(DBTCPConnector.java:404)
at com.mongodb.DBTCPConnector.innerCall(DBTCPConnector.java:238)
at com.mongodb.DBTCPConnector.call(DBTCPConnector.java:216)
at com.mongodb.DBApiLayer$MyCollection.__find(DBApiLayer.java:288)
at com.mongodb.DB.command(DB.java:261)
at com.mongodb.DB.command(DB.java:243)
at com.mongodb.DBCollection.getCount(DBCollection.java:985)

Grails dependency management

June 4, 2013
tags: ,

So for those of you out there who are in grails 2.0 or 2.1 and are not comfortable with the Ivy as build tool for their dependency management because your other projects are using maven, then the good news is Aether which is a library inuse in Maven 3.x – is integrated with grails 2.3 release.

What that implies is a clean transition from ivy to maven without having both co-existing. Also refresh policies can be controlled for your local repositories.

Using maven with grails 2.1 is not a very comfortable option if you try to get out of ivy-cache as all the commands like grails run-app cannot be directly used but have to used through the maven grails plugin. Also we found issues with eclipse not showing maven repository as part of classpath dependencies.

Grails session expiry message using spring security

April 17, 2013

 

Add session expiry parameter’s to logout page – which can be interpreted to show session expired message to user.


securityContextRepository(HttpSessionSecurityContextRepository){}

sessionManagementFilter( SessionManagementFilter, ref("securityContextRepository")){
invalidSessionUrl="/loginPage?parameter=sessionExpired"
}

And in your bootstrap.groovy file add registration of this filter:


def init = { servletContext ->
SpringSecurityUtils.clientRegisterFilter('sessionManagementFilter', SecurityFilterPosition.SESSION_MANAGEMENT_FILTER)

}

 

 

The only issue is that if you want to have a separate message – when the user clicks logout – then this filter needs to be changed. And you need to put some conditions inside it.  SessionManagementFilter is not really extendible – if you see its source code all its field are made either private or final so you need to clone parts of it and put in your own paths to redirect in your custom filter. Also for custom filter – its definition needs to be checked if it should be through web.xml rather than spring security utils class.