Qantas site a mess

[Pushka]

I got the following note

I still can’t log in this morning.

 

[bPeteb]

As this thread likely has attracted some 'IT people', a question.

Is it not possible, or practical to have a 'twin' website/system functioning in every respect but live (and probably based on a 'server' (layman's term) far removed from the 'regular' one). In which case it could:

* Take over after some necessary short period if the main 'server' should go down, or be compromised; and/or

* Take over while the main server website is being maintained/upgraded, allowing the main one to be updated in a non-urgent fashion, and be tested etc?

* When the main is updated, tested and stable, switch it live and after a day or so of functionality, copy it over to the reserve.

... especially for businesses were their main business revenue is generated from their web site?

Hi RooFlyer
I dream of having this. Until bean counters, shareholders and investors give IT departments the budget they should have, so that they can build truly optimal customer experiences, your wonderful and oft discussed solution remains a dream...
bpb

 

[Pushka]

Called up Qantas this morning. They have no idea when it will be fixed.

 

[mrsterryn]

No.app access here either

 

[oz_mark]

As this thread likely has attracted some 'IT people', a question.

Is it not possible, or practical to have a 'twin' website/system functioning in every respect but live (and probably based on a 'server' (layman's term) far removed from the 'regular' one). In which case it could:

There's a bunch of things you could do - especially if you are just making changes to the website (ui/ux changes)

But this case, appears that they have done a data migration of some sort that has gone a little awry.

 

[Ansett]

It is telling me I have no access yet I can enter password and get access to bookings and redeem points for flights. Try just ignoring the banner and enter through as normal.

 

[GoldCanyon340]

While I welcome the addition of the selling class to the Points Activity it is a backward step to have removed the cabin name. I suspect it looks more 'techie' to the casual user. It would be more explanatory to have the cabin name and the selling class in parentheses, ie. Economy (Q).

 

[harvyk]

Well, the scheduled time may not enable all the work to be done in the window. Inevitably, in big systems there are time over-runs or defects that get noticed after deployment that need emergency fix. Often technical bridges where a cast of thousands try to work out what went wrong, and then advising the one person how to fix it. Then That person comes to the end of the shift and goes home so the new tech has to pick up the issue, get their head around it, and sometimes identify what really went wrong and proceed to fix it or in light of the elapsed time roll-back the changes. 9 hours later it's done....or no changes are evident.

If you can imagine it's like trying to do roadworks while the traffic is still using the lane.... ugly at best

No, this has all the hallmarks of a badly managed change. Technical people who are running changes typically don't get to go home mid-way through, you see it out especially if you're the technical lead on the change, and yes that has meant a couple of all nighters for me personally in the past.

The technical lead is suppose to have the technical knowledge to both implement the solution, and directly deal with technical issues which might arise. The sorts of issues which could be encountered during a deployment are supposed to have been pre-identified before release as part of risk assessment matrix, proper risk management and role back plans are supposed to have been done identifying all critical stakeholders for the exact purpose of having people who can just be contacted at any time during the change without swapping and changing people mid-way through.

Since this change has obviously gone over their change window by a long margin, one would have to question why they didn't enact the back out plan before the end of the change control window. The Change Control Window is suppose to provide enough time to implement a change, test it, allow some troubleshooting time, and then allow time for a rollback if required. Failure to allocate enough time to the window is a failure of the Change Control process. This is true of the smallest to the largest of changes.

I'm not going to say this sort of stuff is easy, but they are a large company who should have the resources to get this right, and yet this is yet another example of bad IT management which we've come to expect from good ole QF.

Finally the example of doing roadworks whilst traffic is still in the lane is not exactly a great one. Chances are QF have what are called HA (High Availability) systems. Thus it's possible to turn off a system and let it's backup carry the load whilst the primary is offline. An example of this from my own work is we run a database cluster. I could walk into our server room right now and pull the power plugs from multiple of our production database servers and no one using our systems would be any the wiser. I can also install the bulk of our changes side by side on our production environment that way we can see the changes in and functioning as expected before we flick the switch to use the new code. We're not exactly a big company with loads of resources, so if we can do it, a company as large as Qantas certainly could, if there was the will to do so.

 

[Pushka]

It is telling me I have no access yet I can enter password and get access to bookings and redeem points for flights. Try just ignoring the banner and enter through as normal.

I don’t have the banner. I can enter everything and get the message that they can’t process it right now.

 

[liamg]

hehe

 

[Pushka]

hehe

Does that mean the plane flies in reverse or you sit in a rear facing seat?

 

[rock86]

Can login, and as a NB can now see premium flights 353 days ahead again, woohoo (although I doubt this will last).

But there is a massive issue in that if you're trying to do a multi-city booking, every single fight is getting calculated on an individual basis. It seems there is no combining where there are transits and the flights stay within mile bands, or if you are doing a OWE. Or is this just me?

 

[eminere]

While I welcome the addition of the selling class to the Points Activity it is a backward step to have removed the cabin name. I suspect it looks more 'techie' to the casual user. It would be more explanatory to have the cabin name and the selling class in parentheses, ie. Economy (Q).

Which is already how the booking class is displayed in MMB flight details. Ah the consistency.

 

[Basalt]

Does that mean the plane flies in reverse or you sit in a rear facing seat?

I think a DeLorean at 88 mph is required so that the flights can be unflown back in time...sorry if that makes as much sense as QF's IT change management....

 

[mrsterryn]

It gives a warning on the login page and then when I log in an error message

 

[AisleSeat]

I can't tell you the number of times I've seen or been involved with, as a technical person, absolutely stupid systems changes or implementations because the clueless types in marketing signed off and decreed it should be done

Haha, this.

Our new web template was dreamed up by marketing, who tout it as a, and I quote, "a contemporary and sophisticated digital identity."

No. It's basically the old template with a paint job.

 

[RichardMEL]

btw I am almost certain this change was scheduled for overnight Tues/Wed as it is well known in travel that Tuesday/Wednesday(midweak) are lighter travel days. I onestly think this was scheduled with some thought.

Consider most leisure travel is mostly weekends (which is when many other busineses can do their IT stuff - my window to do big upgrades or migrations etc is after 6pm Friday to affect the least numbers of people)... and business travel is usually Sun eve/Mon headed out and Fri headed back at the end of a work week. Obiously there are going to be exceptions, but midweek travel is generally lighter.

I'm certain this is why it was scheduled for this time.

 

[AisleSeat]

my window to do big upgrades or migrations etc is after 6pm Friday to affect the least numbers of people

Conversely, we avoid doing anything major on a Friday in case something breaks over the weekend.

 

[RichardMEL]

Conversely, we avoid doing anything major on a Friday in case something breaks over the weekend.

it's definitely an issue. One employer I was at (insurance) had a very limited window for changes betwen when one office in one timezone would close and another would open, and we wound up with like a 3am Sunday AEST type timeslot. Not fun.

In terms of the "weekend fail" thing, the thinking in the particular environment is that while services are offered during the weekend, they are limited and the vast majority of staff and sevices are not around between Friday 5-6pm and Monday morning... So, do the work Fri evening BUT if things go pear shaped (or as they say at QF IT, a Wednesday ) then there's more time to fix it, back out, whatever with still the least time effected.

Of course many situations are unique say if involving a 3rd party vendor who may be on the other side of the world doing work, or needing testing from specific teams, or even a decision to change prior to days business opening and that makes the window for "challenges" far smaller and complex, but sometimes those are the requirements.. and sometimes it's gone bad.

it's many of these sorts of issues with complex (and often legacy) systems that make IT not as simple as one might think it should be

and while it's bloody annoying, it also keeps people like me in a job!

 

[GoldCanyon340]

The log in page now recognises the problems with incorrect entries on the points statement.