« previous next »
Pages: [1]   Go Down

Author Topic: cache problem on NSv 9  (Read 3028 times)

Offline evildani

  • Administrator
  • Hero Member
  • *****
  • Posts: 389
  • Karma: 22
cache problem on NSv 9
« on: January 21, 2009, 03:24:01 PM »
We upgraded a netscaler to version 9, and out of the blue we started to have this kind of behavior:

GET /nuevaarp/imagenes/arp_r1_c1.jpg HTTP/1.1
Accept: */*
Referer: http://190.90.32.211/nuevaarp/
Accept-Language: es-co
Accept-Encoding: gzip, deflate
If-Modified-Since: Wed, 29 Oct 2008 16:40:34 GMT
If-None-Match: "0ad4410e539c91:1134"
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Host: 190.x.x.x
Connection: Keep-Alive
Cookie: NSC_bsqxfc_mc=ffffffffaf181c1c45525d5f4f58455e445a4a423660; citrix_ns_id=CZ4DnQdqoUq317M6EB46jbfA7zIA0



HTTP/1.1 400 Bad Request
Content-Type: text/html
Date: Tue, 20 Jan 2009 22:52:36 GMT
nnCoection: close
X-Expires-Orig: None
Pragma: no-cache
Cache-Control: no-cache, must-revalidate
expires: Mon, 26 Jul 1997 05:00:00 GMT
transfer-encoding: chunked

14
<h1>Bad Request</h1>
0

GET /nuevaarp/imagenes/arp_r4_c2.jpg HTTP/1.1
Accept: */*
Referer: http://190.90.32.211/nuevaarp/
Accept-Language: es-co
Accept-Encoding: gzip, deflate
If-Modified-Since: Wed, 29 Oct 2008 16:40:34 GMT
If-None-Match: "0ad4410e539c91:1134"
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Host: 190.x.x.x
Connection: Keep-Alive
Cookie: NSC_bsqxfc_mc=ffffffffaf181c1c45525d5f4f58455e445a4a423660; citrix_ns_id=CZ4DnQdqoUq317M6EB46jbfA7zIA0



HTTP/1.1 200 OK
Xet-Cookie:                                                                                         
Date: Tue, 20 Jan 2009 22:42:46 GMT
Connection: Keep-Alive
Via: NS-CACHE
ETag: "0ad4410e539c91:9e0"
Content-Length: 1225
Content-Type: image/jpeg
Last-Modified: Wed, 29 Oct 2008 16:40:34 GMT
Accept-Ranges: bytes
MicrosoftOfficeWebServer: 5.0_Pub
X-Powered-By: ASP.NET


......JFIF....._._.....C.......................
.
...........................          ...C....
.
.........                                                 ......$.............................................-..........................!.1."AQ.aq$234br.........................................................?..R............................................O3.!...<.
h.....V-Q..Q._...LS...9l.\./a..F..C....w[...]|...F....6...C~..F...X.G.........3so..7.]....AF.........6..c.+..7.M.G.;rq.......r.....hb...s.=.$4.....'.A..w.Z.+..V...x..h..o.f....W...{.....t...........gs%....v.5`=.f.....8..M..0c~egEd...326J...Wn.@... ..s...bWG].B..............m....t.xk.>.W...>..2.G7.
$..`.Z.\.%d.k...88.7.4....A...............?P...`rX....U..8...2.....k.....?Z..~.V...|K.~O.6..d{n.D}$t....l.......}Y..l
..p.....c.....-.;z!...j[.?. ......D[.o.....=x. v.c...~......`kE.8.a.....a.;.n...V...cP...Cr.xI.[..g..E..i.p\.Pru.B...L
4hLd.....g%e.g...<pc....!...].M.l....jW'..../.d.n.$......\y.....e......f.....b...u....r`.$^2..x..] ......o.^.[.....|j.......].s.?..........tT<.?...)E.bS..2]....dw.
4.A&.....>.mu...W...uW...s..q.4.....b,b'.Wb.~..+&.7....K..n.^.................#.w..c.@ ......@@A.
.....h>..v...lF...J........................................
Logged

Offline evildani

  • Administrator
  • Hero Member
  • *****
  • Posts: 389
  • Karma: 22
Re: cache problem on NSv 9
« Reply #1 on: January 28, 2009, 02:56:15 PM »
Bug 54486 in 9.0 release.
Due to this bug, the cache contents on the NS get corrupted, which causes the NS to change the header If-None-Match: to an incorrect value. This in turn causes the server to return a HTTP 400 (Bad request) response  or no response at all.
 
The bug has already been fixed and checked in on Jan 19,2009 and will be available in a GA release of 9.0(9.0 b 68.x) scheduled to be released in mid April 2009.
 
The following workaround can be used meanwhile. To implement the workaround, please do the following:
1. Flush the NS cache  by running the command in CLI
  flush cache contentgroup all

2. Run the following command in shell. This will turn off a new feature in 9.0, which compacts the cache - this feature causes corruption of the cache.
    nsapimgr -B "call ns_cache_set_compact_cells(0)"
Logged

Offline pathnetworks

  • Sr. Member
  • **
  • Posts: 22
  • Karma: 1
Re: cache problem on NSv 9
« Reply #2 on: February 10, 2009, 08:17:55 AM »
Thanks!

The system has been extremely stable for me otherwise, but the cache problem has been an issue.    This really helps to cover the problem until the bug fixin' later.

Logged

Offline pathnetworks

  • Sr. Member
  • **
  • Posts: 22
  • Karma: 1
Re: cache problem on NSv 9
« Reply #3 on: February 11, 2009, 11:41:43 AM »
Quote from: evildani on January 28, 2009, 02:56:15 PM
Bug 54486 in 9.0 release.
<SNIP>
The bug has already been fixed and checked in on Jan 19,2009 and will be available in a GA release of 9.0(9.0 b 68.x) scheduled to be released in mid April 2009.
 
The following workaround can be used meanwhile. To implement the workaround, please do the following:
1. Flush the NS cache  by running the command in CLI
  flush cache contentgroup all

2. Run the following command in shell. This will turn off a new feature in 9.0, which compacts the cache - this feature causes corruption of the cache.
    nsapimgr -B "call ns_cache_set_compact_cells(0)"

Shoot, the new release 9.0 Build 66.12 that was available since 2/8/2009 still doesn't fix this according to the release notes.    Or at least there is no metion of Bug 54486.    Guess we'll still have to wait until mid April.

A huge number of issues have been fixed (60 in all for this specific release), it may still be wise to wait for Netscaler customers on 8.1 or earlier until the April release is available.      However, for those that have 9.0 already, I'd recommend reviewing the newer 66.12 right away.     Remeber, reviewing isn't immediatly installing on production but testing in a lab if possible.   Not everyone can avoid that however and a full plan should be considered before deployment.
Logged

Offline ZManGT

  • VIP Member
  • ***
  • Posts: 94
  • Karma: 12
Re: cache problem on NSv 9
« Reply #4 on: February 12, 2009, 04:50:07 AM »
This is off topic but I must say I'm really disappointed in the QA effort that went into this build. It appears to have many bugs that should have been noticed and were either over looked or ignored. Hopefully Citrix steps up their effort and starts to QA these devices a little more thoroughly in the future.
Logged

Offline jmelika

  • Administrator
  • Hero Member
  • *****
  • Posts: 341
  • Karma: 7
Re: cache problem on NSv 9
« Reply #5 on: February 12, 2009, 10:35:48 AM »
I have another off topic question.  Where do you guys keep coming up with spare Netscaler boxes to use in a lab environment?  Does Citrix give those out if you ask for them or what?  I'm missing out since I normally do my changes straight on my live environment with my fingers crossed and my stuff in a box just in case. :)
Logged

Offline ZManGT

  • VIP Member
  • ***
  • Posts: 94
  • Karma: 12
Re: cache problem on NSv 9
« Reply #6 on: February 13, 2009, 06:06:20 AM »
Jmelika,

We got the "box" for free because of purchasing problems with all of our equipment. However we still have to buy the maintenance, support, and all the modules we need for it, and to be honest I don't think we get any discount for it as a test box over our standard agreement with Citrix.
Logged

Offline pathnetworks

  • Sr. Member
  • **
  • Posts: 22
  • Karma: 1
Re: cache problem on NSv 9
« Reply #7 on: February 13, 2009, 11:08:28 AM »
Quote from: jmelika on February 12, 2009, 10:35:48 AM
I have another off topic question.  Where do you guys keep coming up with spare Netscaler boxes to use in a lab environment?

It's the "spare" that we purchased on hand.   Even with most vendors "instant replacement warranties", you still wait a day for the overnight shipping.    We planned one on hand in the cost of the project.    Not everyone can have that, but if possible it's always recommended to not test in production.    You may still have to cross your fingers, but you don't have to prepack a box in your office.
Logged

Offline evildani

  • Administrator
  • Hero Member
  • *****
  • Posts: 389
  • Karma: 22
Re: cache problem on NSv 9
« Reply #8 on: February 13, 2009, 12:04:16 PM »
I use "Not For Resell" units. I work for a Citrix partner so we get each box really cheap... The down side is that they are 7000 units...
I am trying to get my hand on one for my home... lets see how that works out :(
Logged

Offline Marco Schirrmeister

  • Hero Member
  • *****
  • Posts: 101
  • Karma: 14
Re: cache problem on NSv 9
« Reply #9 on: February 20, 2009, 04:45:38 PM »
jmelika,

our spare boxes are from some internal infrastructure changes where implemented new devices.
And since we have many important applications running we were able to convince our boss that it is good to keep testing devices.

In my eyes and what I noticed in the past it's really helpful to have a device where you can play with. This are older legacy 9800 devices. But they still perform very good.


Marco


Logged

Offline pathnetworks

  • Sr. Member
  • **
  • Posts: 22
  • Karma: 1
Re: cache problem on NSv 9
« Reply #10 on: March 23, 2009, 01:21:31 PM »
Noticed the new release is available now.

NS_9_0_67_7:

Specifically mentions the Integrated Caching fix:

52) Integrated Caching (Issue ID 54486)

Noticed that there are a total of 71 Bug Fixes in this release and 16 "Enhancements".

I'll be patching soon when I get it into the schedule.    Anyone else have this on their patching scope?   I'm hopefull that this issue is resolved, but I'll know soon.
Logged

Offline jmelika

  • Administrator
  • Hero Member
  • *****
  • Posts: 341
  • Karma: 7
Re: cache problem on NSv 9
« Reply #11 on: March 23, 2009, 04:39:02 PM »
Not on my roadmap, but I'd love to know how it works out for you.  Please keep us posted.
Logged

Offline evildani

  • Administrator
  • Hero Member
  • *****
  • Posts: 389
  • Karma: 22
Re: cache problem on NSv 9
« Reply #12 on: March 23, 2009, 04:58:38 PM »
I will be upgrading two clients of mine this week. If it solves the cache problem it will be enough.

Daniel
Logged

Offline pathnetworks

  • Sr. Member
  • **
  • Posts: 22
  • Karma: 1
Re: cache problem on NSv 9
« Reply #13 on: March 27, 2009, 03:28:30 PM »
So far so good!

Appears to have solved the cache problem.

Make sure that if you followed Evildani's work arround to reverse the configuration so the feature is once again enabled.

    nsapimgr -B "call ns_cache_set_compact_cells(1)"     

(0) was the work arround as it disabled the feature.


Logged

Offline pathnetworks

  • Sr. Member
  • **
  • Posts: 22
  • Karma: 1
Re: cache problem on NSv 9
« Reply #14 on: May 04, 2009, 09:04:34 AM »
My biggest complaint with the prior release was a bug with symbolic links (used for customized Access Gateway functions).

This has finally been fixed with the latest release 68.6.

I'm using this now.    I'll report back in a week to let you know the results.
Logged
Pages: [1]   Go Up
« previous next »