Skip to content

Depth

October 4, 2016

So unless you put your heart, soul and mind and focus in only one and only one thing which is learning it is difficult to speak the ease with depth of an architect.

How can one develop this focus?  So instead you are asking how does one develop ekagrita – one chitta – one attention for going deep – developing DEPTH.

Depth requires focus, focus requires discarding, and doing only one thing in life. To learn in the right direction.

Guava Mongodb Caching in Grails

July 3, 2016

Basic cache expiration

Expiration in Guava happens when few specific methods (like get/size etc ) are called. It does happen with a timer.So if you set it to 5 mins, expiry may happen few seconds later , when next get comes for same key.

Here is what happens.
Thread A: Get the key. Key has expired, returns false, get lock on cachekey, store data (Data is now doubled as delete on eviction has not yet run)
Thread B: Eviction has taken place, all data gets deleted, data is now 0 in cache .

We used Locks for synchronized access. Synchronize guarantees one thread accesses 1 op at a time, but what it does not guarantee is the Sequence of our operations.
So we want eviction to complete before storage happens, or we want eviction to not delete the newly stored entities. But we only have cacheKey as identifier so how do we differentiate !.

I even checked the mongodb eviction policy but that is also not immediate – it runs every 60 seconds/ depending on load etc. and if immediate eviction with continuous polling or timer can be a costly operation both at the db end as well in the guava cache library, the only option out that is quick and easy is to keep timestamp.

That can only happen when timestamp is maintained in the entities. I remembered that timestamps can be created in domain objects.
But keeping timestamp in all the DO’s is meaningless because we don’t want to know time DO is created but the time cache is created, so what we really want is to keep timestamp in the cacheSignature.

So here could be the scenarios:
· Timestamp is created when the cache is inserted in DB & entry is put in map , so the stored DO’s will have timestamp field but all DO’s will have the same time stamp value indicative of when cache was created. This timestamp is inserted in cacheSignature prior to cacheMap entry put call.
· When get happens, those entries which are older than expiry time than current will not be fetched.
· When delete happens , those entries which are expired as per current timestamp get deleted.

At the time of get/and store/ and delete we use this timestamp feature to fetch selective data.
The other advantage of this approach over writing your own eviction code based on the timestamp is that the getter code will not need to wait for delete Operation to complete before returning results to user.

ElasticSearch

July 1, 2016

ES problem was handling of relationships:

1. handle it at app level – hit two different indices. with ID’s
2. Nested children – problem is re-indexing of parent every time children are updated, mapping has to be of nested type. costly if too many children
3. Parent Child – can do : parent indexed separately and children indexed separately. link is mantained between them.
children can be queried separately. need not tranverse thru parent

Char Filter-> Tokenizer -> Token Filter

term/ term(s) : not analyzed

wildcards
filters
missing
must_not
query_string
range
field{ gte, lte }

The term query does not analyze the search term, the match query uses the same analyzer for search as was used to index the field

query: filtered : [query, filter ]
query : match_phrase or query :match : field { query: value, type:match_phrase}
query: must : match : field
query : bool or query->filtered->filter:bool
query:fitleredXXXbool does not work
query_string

query
filtered filter OR / AND / EXISTS/

filtered
query
match, latest

must is keep score count, should does not add to score count
term does not use analyzer, must uses the default analyzer,

match_phrase query is the one you should reach for when you want to find words that are near each other:

{
    "query_string" : {
        "fields" : ["city.*","name","age"],
        "query" : "this AND that OR thus",
        "use_dis_max" : true
    }
}

query -> match_phrase

 

multi field mapping with two mappings – one is for existing token other is not analyzed

"title": {
    "type": "multi_field",
    "fields": {
        "title": { "type": "string" },
        "raw":   { "type": "string", "index": "not_analyzed" }
    }
}

or

"title": {
    "type": "string",
    "fields": {
        "raw":   { "type": "string", "index": "not_analyzed" }
    }
}

Kyle’s JS – Points

March 3, 2016

If strict mode is in effect, the global object is not eligible for the default binding, so the this is instead set to undefined:

this Binding Rules 4: default (global) , implicit (fn call) , explicit (bind, call apply) , new binding.

Sequence is: 1. new fn call, 2.call/apply/bind 3. obj.fn call 4. default

safe object:

// our   empty object
var ø = Object.create( null );

Concept of soft binding so that global object does not get used ever !!

 

obj1.obj2.foo means foo is called in context of obj2

if you do bar = o1.o2.foo and then you call bar() – bar is called without context – global or undefined in case of strict

Explicit binding is when you use call or apply and pass the first parameter – as the object context.wrap a function with a hard binding

var bb = function() { foo.call(obj); }

bb(); // in case of global or set timeout, will always use the right context for calling object.

By calling foo(..) with new in front of it, we’ve constructed a new object and set that new object as the this for the call of foo(..). So new is the final way that a function call’s this can be bound. We’ll call this new binding.

bind(..) returns a new function that is hardcoded to call the original function with the this context set as you specified.

obj1.foo.call( obj2 ); // 3
obj2.foo.call( obj1 ); // 2

So, explicit binding takes precedence over implicit binding, which means you should ask first if explicit binding applies before checking for implicit binding.

 

 

HTTPS communication jdk version

March 2, 2016

Defaults of ssl protocol differ in each jdk .

Add the property -Dhttps.protocols=TLSv1 in tomcat shell script to ensure same version used in both.  Preferably keyalg=RSA and keysize=2048

The following chart depicts the protocols and algorithms supported in each JDK version:

JDK 8
(March 2014 to present)
JDK 7
(July 2011 to present)
JDK 6
(2006 to end of public updates 2013)
TLS Protocols TLSv1.2 (default)
TLSv1.1
TLSv1
SSLv3
TLSv1.2
TLSv1.1
TLSv1 (default)
SSLv3
TLS v1.1 (JDK 6 update 111 and above)
TLSv1 (default)
SSLv3
JSSE Ciphers: Ciphers in JDK 8 Ciphers in JDK 7 Ciphers in JDK 6
Reference: JDK 8 JSSE JDK 7 JSSE JDK 6 JSSE
Java Cryptography Extension, Unlimited Strength (explained later) JCE for JDK 8 JCE for JDK 7 JCE for JDK 6

mod_pf and integration with PF SP

February 11, 2016

mod_pf setup is straightforward – module so and mod_pf conf file in along with agent_config (These 2 config files taken from SP).

However there were few issues that came up.

  1. There is token renew timeout (5 minutes by default) which is re-created based on active session. This timeout – for xhr calls was not followed for redirection, as XHR calls do not follow until the response header from the server says Access-Control-Allow-Origin: * or the domains where xhr originated from.

To resolve this the PF SP should update the file

pingfederate\server\default\data\config-store\response-header-runtime-config.xml

set header value for Access-Control-Allow-Origin : include domain here.

similarly for X-Frame-Options you can set as sameorigin.

Without these headers the xhr request will not get followed up.

2. Session timeout : Now renew-util is maximum time for which session can be active. Not the max inactive time. Note the difference, so if you say renew-until 2 hours,after 2 hours even if user is active, he will be pushed out🙂 or to rephrase it PF does not have a way to track inactive session period.  Concept of SLO does exist in PF , however for our env it was not enabled, so we had to rely on app session timeout variance.

3. Logout – url needs to have a post action to push to apache level logout that pushes to PF logout page where all cookies are cleared up and individually calls sub app’s logout url.

4. Since its a header based auth – for security reason no app should be accessible directly without the proxy.

 

Gawk and upload of data in neo4j

March 25, 2015