Anyone know how to Analyze "crashinfo" files from Cisco access points?
Anyone know how to Analyze "crashinfo" files from Cisco access points?. Discuss Anyone know how to Analyze "crashinfo" files from Cisco access points?, on Wireless Forums.
Anyone know how to Analyze "crashinfo" files from Cisco access points?
Hello ...
We have a number of Cisco AP350 wireless access points thoughout our
network (as well as some AP1100s and AP1210s, but these aren't involved
in the problem that's causing me to send this message).
One of them seems to have started periodically crashing and resetting
this past Monday evening (2006/08/28, 21:57:51, according to the
timestamp on the crashinfo file), and is presently apparently looping
between functioning and not. I have 10 crashinfo files and an r0.rcore
file from it, and would like to try and figure out what the problem is.
Can anyone point me to information regarding how these should be used
for crash analysis?
The syslog from this device is presently cycling through the following
messages. I think I'm going to disconnect the device at this point ...
Aug 30 12:40:21 ap-gm401-90 573: Aug 30 12:40:21: %DOT11-2-RADIO_FAILED:
Interface Dot11Radio0, failed - cmd busy
Aug 30 12:40:21 ap-gm401-90 574: -Traceback= 4DC04 4511C4 45110C 4532B0
6AA4DC 6AA778 6AF00C 4503EC 4507C8 450B1C 1529D8 157988
Aug 30 12:40:22 ap-gm401-90 575: Aug 30 12:40:21: %LINK-3-UPDOWN:
Interface Dot11Radio0, changed state to down
Aug 30 12:40:22 ap-gm401-90 576: Aug 30 12:40:22: %LINEPROTO-5-UPDOWN:
Line protocol on Interface Dot11Radio0, changed state to down
Aug 30 12:40:46 ap-gm401-90 577: Aug 30 12:40:45: %LINK-5-CHANGED:
Interface Dot11Radio0, changed state to reset
Aug 30 12:40:48 ap-gm401-90 578: Aug 30 12:40:47: %LINK-3-UPDOWN:
Interface Dot11Radio0, changed state to up
Aug 30 12:40:48 ap-gm401-90 579: Aug 30 12:40:48: %LINEPROTO-5-UPDOWN:
Line protocol on Interface Dot11Radio0, changed state to up
Systems and Network analyst Concordia University
Instructional & Information Technology Montreal, Quebec, Canada
----------------------------------------------------------------------
>Hello ...
>
>We have a number of Cisco AP350 wireless access points thoughout our
>network (as well as some AP1100s and AP1210s, but these aren't involved
>in the problem that's causing me to send this message).
>
>One of them seems to have started periodically crashing and resetting
>this past Monday evening (2006/08/28, 21:57:51, according to the
>timestamp on the crashinfo file), and is presently apparently looping
>between functioning and not. I have 10 crashinfo files and an r0.rcore
>file from it, and would like to try and figure out what the problem is.
>
>Can anyone point me to information regarding how these should be used
>for crash analysis?
>
>The syslog from this device is presently cycling through the following
>messages. I think I'm going to disconnect the device at this point ...
>
>Aug 30 12:40:21 ap-gm401-90 573: Aug 30 12:40:21: %DOT11-2-RADIO_FAILED:
> Interface Dot11Radio0, failed - cmd busy
>Aug 30 12:40:21 ap-gm401-90 574: -Traceback= 4DC04 4511C4 45110C 4532B0
> 6AA4DC 6AA778 6AF00C 4503EC 4507C8 450B1C 1529D8 157988
>Aug 30 12:40:22 ap-gm401-90 575: Aug 30 12:40:21: %LINK-3-UPDOWN:
> Interface Dot11Radio0, changed state to down
>Aug 30 12:40:22 ap-gm401-90 576: Aug 30 12:40:22: %LINEPROTO-5-UPDOWN:
> Line protocol on Interface Dot11Radio0, changed state to down
>Aug 30 12:40:46 ap-gm401-90 577: Aug 30 12:40:45: %LINK-5-CHANGED:
> Interface Dot11Radio0, changed state to reset
>Aug 30 12:40:48 ap-gm401-90 578: Aug 30 12:40:47: %LINK-3-UPDOWN:
> Interface Dot11Radio0, changed state to up
>Aug 30 12:40:48 ap-gm401-90 579: Aug 30 12:40:48: %LINEPROTO-5-UPDOWN:
> Line protocol on Interface Dot11Radio0, changed state to up
>
>
>Any ideas would be greatly appreciated ...
>
>--
>----------------------------------------------------------------------
>Sylvain Robitaille syl@alcor.concordia.ca
>
>Systems and Network analyst Concordia University
>Instructional & Information Technology Montreal, Quebec, Canada
>----------------------------------------------------------------------
Re: Anyone know how to Analyze "crashinfo" files from Cisco access points?
gene martinez wrote:
> Did you contact Cisco????
>
> Sylvain Robitaille <syl@alcor.concordia.ca> wrote:
>
> >Hello ...
> >
> >We have a number of Cisco AP350 wireless access points thoughout our
> >network (as well as some AP1100s and AP1210s, but these aren't involved
> >in the problem that's causing me to send this message).
> >
> >One of them seems to have started periodically crashing and resetting
> >this past Monday evening (2006/08/28, 21:57:51, according to the
> >timestamp on the crashinfo file), and is presently apparently looping
> >between functioning and not. I have 10 crashinfo files and an r0.rcore
> >file from it, and would like to try and figure out what the problem is.
> >
> >Can anyone point me to information regarding how these should be used
> >for crash analysis?
> >
> >The syslog from this device is presently cycling through the following
> >messages. I think I'm going to disconnect the device at this point ...
> >
> >Aug 30 12:40:21 ap-gm401-90 573: Aug 30 12:40:21: %DOT11-2-RADIO_FAILED:
> > Interface Dot11Radio0, failed - cmd busy
> >Aug 30 12:40:21 ap-gm401-90 574: -Traceback= 4DC04 4511C4 45110C 4532B0
> > 6AA4DC 6AA778 6AF00C 4503EC 4507C8 450B1C 1529D8 157988
> >Aug 30 12:40:22 ap-gm401-90 575: Aug 30 12:40:21: %LINK-3-UPDOWN:
> > Interface Dot11Radio0, changed state to down
> >Aug 30 12:40:22 ap-gm401-90 576: Aug 30 12:40:22: %LINEPROTO-5-UPDOWN:
> > Line protocol on Interface Dot11Radio0, changed state to down
> >Aug 30 12:40:46 ap-gm401-90 577: Aug 30 12:40:45: %LINK-5-CHANGED:
> > Interface Dot11Radio0, changed state to reset
> >Aug 30 12:40:48 ap-gm401-90 578: Aug 30 12:40:47: %LINK-3-UPDOWN:
> > Interface Dot11Radio0, changed state to up
> >Aug 30 12:40:48 ap-gm401-90 579: Aug 30 12:40:48: %LINEPROTO-5-UPDOWN:
> > Line protocol on Interface Dot11Radio0, changed state to up
> >
> >
> >Any ideas would be greatly appreciated ...
Ther eare tools on the Cisco web site but I am pretty sure that you
need
a login.
Re: Anyone know how to Analyze "crashinfo" files from Cisco access points?
Merv wrote:
> There is a traceback message amongst those you posted.
Yes, and a number of them in the crashinfo files as well.
> A traceback almost invariably means a bug.
Thanks, I imagined as much. Incidentally, I turned the radio off on
this unit, and it appears to have stopped crashing. I'm not sure
whether that helps pinpoint the problem in hardware or software, but I
find it interesting.
Systems and Network analyst Concordia University
Instructional & Information Technology Montreal, Quebec, Canada
----------------------------------------------------------------------
Systems and Network analyst Concordia University
Instructional & Information Technology Montreal, Quebec, Canada
----------------------------------------------------------------------
Systems and Network analyst Concordia University
Instructional & Information Technology Montreal, Quebec, Canada
----------------------------------------------------------------------
Re: Anyone know how to Analyze "crashinfo" files from Cisco access points?
Merv wrote:
> One of the first things the Cisco TAC will ask you to do is to upgrade
> to 12.3(8)JA2
Sigh. That doesn't surprise me if the new version is available; there
is some possibility that it will resolve the problem, given the symptom
strongly suggests "software bug".
> Do you have other AP350 running this release?
Another 112 of them, yes.
> How long had the AP350 with the problem been running with 12.3(8)JA?
Months. I don't recall exactly when I performed the upgrade (I do have
that recorded somewhere, though I'm not at work as I write this), but it
was late March.
> What release was it running before that ?
Based on archived configuration files, I can say for sure that it was
running IOS, but prior to upgrading all the access points I had numbers
of access points running different versions of IOS, and some running
different versions of VxWorks (if that doesn't suggest "inherited
installation" to you, allow me to disclaim that "it was like that when I
got it!"). I don't recall having made an exact list then of which
access points were running what software versions (and have already
wished I had ...)
Once I had a good grasp of the size of the installation, and the different
models of equipment we have (AP350s, AP1100s, and AP1210s), with which
software versions we were running, I undertook first to convert all
VxWorks access points to IOS, then to upgrade all access points to the
then-latest version of IOS. I've had very little trouble since (two
access points that were routinely power-cycling prior to upgrades have
still been exhibiting that behaviour since). The problem I'm reporting
here is very sudden.
I'll have a look for 12.3(8)JA2, and peruse release notes ...
Systems and Network analyst Concordia University
Instructional & Information Technology Montreal, Quebec, Canada
----------------------------------------------------------------------
Re: Anyone know how to Analyze "crashinfo" files from Cisco access points?
One thing to consider for future is to keep in mind that Cisco usually
releases 3 or 4 minor maintenace releases. This varies by the target
technology.
It is not usually the the first one to be deferred ( read junked):
12.3.8-JA2 (ED)
12.3.8-JA (ED)
12.3.7-JA3 (ED)
12.3.7-JA2 (ED)
12.3.7-JA1 (ED)
12.3.4-JA2 (ED)
12.3.4-JA1 (ED)
12.3.4-JA (ED)
12.3.2-JA6 (ED)
12.3.2-JA5 (ED)
12.3.2-JA2 (ED)
I like to wait for the 3rd or 4th of these maintenance releases.
So I would suggest you wait for awhile before considering updating your
entire fleet (again).
Re: Anyone know how to Analyze "crashinfo" files from Cisco access points?
> > Did you contact Cisco????
>
> I imagine I'll need to do that, but would like to have some idea of the
> problem first. Thanks, though.
Where this dlink or some other vendor with "free" support that'd have merit.
But if you've got that many cisco units you'd probably be better off calling
them FIRST instead of waiting for the usenet rabble to chime in...
Systems and Network analyst Concordia University
Instructional & Information Technology Montreal, Quebec, Canada
----------------------------------------------------------------------
Re: Anyone know how to Analyze "crashinfo" files from Cisco access points?
Merv wrote:
> For 12.3(8)JA2, take a close look at CSCsb00606
>
> You might be better to drop back to 12.3(7)JA3 on the troubled unit.
>
> Also note that the first maintenance release usually gets deferred (
> read junked):
>
> 12.3.8-JA2
>
> 12.3.7-JA3
> 12.3.7-JA2
> 12.3.7-JA1
>
> 12.3.4-JA2
> 12.3.4-JA1
> 12.3.4-JA
>
> 12.3.2-JA6
> 12.3.2-JA5
> 12.3.2-JA2
>
> I normally like to wait for the third or fourth maintenenace release
> depending on the technology.
I may have the wrong end of the stick here, do you mean for
example 12.3.2-JA4 is the fourth maint. rel?
I am sure that Merv has more Cisco experience than I have
however, I do not like the idea of what amount to unplanned
releases. ...JAx where x = 1, 2, 3, ... are I understand
fixes that are applied to software that is supposed to be working.
I fancy releases that do not need so much fixing.
12.3.2-JA6 just makes my skin crawl.
As I say this is only a wild guess with absolutely
no evidence either way. I just feel that it looks like a Friday
afternoon product. Maybe it can be fixed, but why not just
stay away if that is an option?
Re: Anyone know how to Analyze "crashinfo" files from Cisco access points?
Well in the "old days" before IOS version "trains" the number in
between the ( ) was the maintenance release - things were much simpler
than.
12.3JA is known as a STED - specific technology early deployment
release - and as far as I can tell the number after the JA sure looks
and feels like a maintenance as one can tell by looking carefully at
the release notes.
Re: Anyone know how to Analyze "crashinfo" files from Cisco access points?
On Thu, 31 Aug 2006 14:51:39 +0000 (UTC), Sylvain Robitaille <syl@alcor.concordia.ca> wrote:
~ Merv wrote:
~
~ > For 12.3(8)JA2, take a close look at CSCsb00606
~
~ Hrmmm... Thanks for that.
CSCsb00606 was actually filed against 12.3(2)JA. It doesn't look like
anything's happened with that one in a year. I don't suppose it would
pertain to current IOS.
~ > You might be better to drop back to 12.3(7)JA3 on the troubled unit.
~
~ Will have a look. Thanks again.
Both 12.3(7)JA3 and 12.3(8)JA2 are good solid releases. Btw, we just
posted 12.3(8)JEA today, if you really want to run the latest and
greatest.
That said, it's not clear to me (after scanning known bugs) whether
the condition you encountered that triggered the radio failure is going
to be addressed by upgrading software.
Re: Anyone know how to Analyze "crashinfo" files from Cisco access points?
Merv wrote:
> Well in the "old days" before IOS version "trains" the number in
> between the ( ) was the maintenance release - things were much simpler
> than.
>
> 12.3JA is known as a STED - specific technology early deployment
> release - and as far as I can tell the number after the JA sure looks
> and feels like a maintenance as one can tell by looking carefully at
> the release notes.
To original question - try this website. I know the proprietor is on
holiday from Monday for a couple of weeks but might respond after this.
Re: Anyone know how to Analyze "crashinfo" files from Cisco access points?
Merv wrote:
> Well in the "old days" before IOS version "trains" the number in
> between the ( ) was the maintenance release - things were much simpler
> than.
>
> 12.3JA is known as a STED - specific technology early deployment
> release - and as far as I can tell the number after the JA sure looks
> and feels like a maintenance as one can tell by looking carefully at
> the release notes.
I think that it is like this:-
a.b(cx)JAy
JA tells us that it is a special (required for for many purposes)
Avoid specials of you can
These releases are based on a mainstream release
in this case a.b(cx).
a - Major release
b - Minor release
(c) - Maintenance release - in some trains can contain new features
x - Re-release of a.b(c) [On which JA is based]
y - Re-release of a.b(c)JA - changes not affecting base a.b(c)
x is always a lower case letter
y is always a number
As I say, I worry about re-releases but have no direct IOS derived
evidence on which to base it.
Re: Anyone know how to Analyze "crashinfo" files from Cisco access points?
Aaron Leonard wrote:
> Both 12.3(7)JA3 and 12.3(8)JA2 are good solid releases.
In my experience, that's also been true of 12.3(8)JA, save for a small
number of units that have been acting up (the one we're discussing here
that failed only recently, and a couple of others that have been acting
up since before the upgrade to 12.3(8)JA).
> Btw, we just posted 12.3(8)JEA today, if you really want to run the
> latest and greatest.
No, I'm more interested in running the same version across all our access
points. Only when it becomes necessary to upgrade (due to serious bugs,
or specific new features, or as I did earlier this year, to bring all
our units up to the same version) do I start looking at upgrading to
the most current version (or ${most_current}-1, if ${most_current}
hasn't been around long enough to get a bit of a shakedown).
> That said, it's not clear to me (after scanning known bugs) whether
> the condition you encountered that triggered the radio failure is going
> to be addressed by upgrading software.
You're probably right. We've started discussing replacing that unit
with a newer (AP1200) one. That's very likely what I'll do.
Systems and Network analyst Concordia University
Instructional & Information Technology Montreal, Quebec, Canada
----------------------------------------------------------------------
Anyone know how to Analyze "crashinfo" files from Cisco access points? 
Linear Mode
