I've been a believer, but just been cheap (or frugal). But it probably won't get any better than this. I might pull the trigger
Amazon Prime Day 2018 Deals (for Prime members, ENDS 7/17 at 11:59P EST)
- Thread starter a1rimrocka
- Start date
the cheapest I’ve ever seen it is $149.99
Just talked with customer service about my issues and got a another 15% off.
Nvidia Shield deal is live for $139.00. Deal is 15% claimed as of this post, ends at 8:45pm EST tonight
https:// www.amazon.com/NVIDIA-SHIELD-Streaming-Player-Remote/dp/B075RXV2VR
https:// www.amazon.com/NVIDIA-SHIELD-Streaming-Player-Remote/dp/B075RXV2VR
Beware of those unofficial Android boxes, sometimes u get what u pay for. And there's a chance YoutubeTV may not even install or run on it.
They advertise as being 4k-capable but certain apps (Netflix, YouTube, etc.) don't run or allow 4k if the box isn't certified.
For YTTV your safest bet in that price range now is a Roku Stick. Next best option is a Mi Box (~$60) if you can find it or bite the bullet on that Shield for $139
I'm noticing that this offer doesn't include the gaming Controller. Not a problem since I'm not a gamer, but is this the latest, smaller version of the shield?
Last edited:
If you have a controller that uses Bluetooth you good
Yep. PS3, Xbox One S controllers work great on my Shield TV
Only thing i've gotten so far has been a couple bedroom comforter sets, a phone stand for my whip (they bussin heads with those hands free tickets in GA), and some cleaning supplies. Haven't really seen anything else that moved the needle for me.
This kind of thing keeps me shopping with Amazon for everything
Last week I complained that their delivery man lied about delivering a pair of shoes directly to me and they just refunded me for the shoes, no return necessary. Their customer service >>>
Copped the Shield and forgot I had a $35 credit/gift card I've been saving for something good. Only issue is they must have got hit hard because it's saying up to three weeks for delivery
I thought i missed missed it this afternoon and bought it from ebay for $30 more. Check out the link....I hope this helps someone
https://amp-hothardware-com.cdn.amp...ware.com/news/ebay-amazon-prime-day-25-coupon
https://amp-hothardware-com.cdn.amp...ware.com/news/ebay-amazon-prime-day-25-coupon
Just ordered my 4k fire stick!
Any good prime day deals on home surveillance cameras?
This deal is pretty good at Best Buy... Turn in an old streaming device and get $30 off a Shield, making the one with remote & controller only $149 (while it's on sale). Looks like this trade-in promo is good til August 11
https://www.bestbuy.com/site/promo/nvidia-shield-in-store-offer
Last edited:
SWEET! I'll see if I can trade in my old Roku 2
Didn't even think about that ... is there a thread on this?
there have been several...just do a search for ‘nvidia shield’ @EPDC has made a couple threads and quite a few posts @MurderCity provided detailed instructions on how to add emulators...I’ll see if I can find it
OG nvidia shield thread...
https://www.bgol.us/forum/threads/tech-heads-nvidia-shield-new-media-streaming-gaming-device.850869/
I’ve bought it on 3 different occasions...I returned it a couple times thinking I could manage w/ out it lol...but it really is the ultimate media device...I currently have the 16GB and use a 1TB external drive w/ it...I like using the native game controller w/ it since I use kodi/SPMC and it offers more features
https://www.bgol.us/forum/threads/tech-heads-nvidia-shield-new-media-streaming-gaming-device.850869/
I’ve bought it on 3 different occasions...I returned it a couple times thinking I could manage w/ out it lol...but it really is the ultimate media device...I currently have the 16GB and use a 1TB external drive w/ it...I like using the native game controller w/ it since I use kodi/SPMC and it offers more features
Last edited:
I just got the Nvida Shield device. I know my girlfriend is going to be excited once I hook it up and get it rolling.
FYI these are still available for sale price picked up today.This deal is at LOWES right now (for Prime Day only) - $99.99, normally $300
Samsung 3-Pack Connect Home Smart Wi-Fi System AC1300
https://www.lowes.com/pd/Samsung-3-Pack-Connect-Home-Smart-Wi-Fi-System-AC1300/1000256453
Picked this up on best buys web site a month ago at the same price. Thought it was on speacial cause an updated version was coming but these joints work so well, what else can you do to make it better?
Internal documents show how Amazon scrambled to fix Prime Day glitches
Eugene Kim | @eugenekim222
Published 2:43 PM ET Thu, 19 July 2018Updated 6:16 PM ET Thu, 19 July 2018CNBC.com
Amazon.com founder and CEO Jeff Bezos
Amazon failed to secure enough servers to handle the traffic surge on Prime Day, causing it to launch a scaled-down backup front page and temporarily kill off all international traffic, according to internal Amazon documents obtained by CNBC.
And that took place within 15 minutes of the start of Prime Day — one of Amazon's biggest sales days every year.
The e-commerce giant also had to add servers manually to meet the traffic demand, indicating its auto-scaling feature may have failed to work properly leading up to the crash, according to external experts who reviewed the documents. “Currently out of capacity for scaling,” one of the updates said about the status of Amazon’s servers, roughly an hour after Prime Day’s launch. “Looking at scavenging hardware.”
A breakdown in an internal system called Sable, which Amazon uses to provide computation and storage services to its retail and digital businesses, caused a series of glitches across other services that depend on it, including Prime, authentication and video playback, the documents show.
Other teams, including Alexa, Prime Now and Twitch, also reported problems, while some warehouses said they weren’t even able to scan products or pack orders for a period of time.
The documents give a rare look into how Amazon responded to the higher-than-expected traffic surge on Prime Day, which caused glitchesacross the site for hours. It also illustrates the difficulty Amazon faced in dealing with the demand, despite its deep experience running a massive-scale website and one of the largest cloud computing platforms in the world.
“More people came in than Amazon could handle,” Matthew Caesar, a computer science professor at the University of Illinois and co-founder of cybersecurity firm Veriflow, said after CNBC shared the details of the documents. “And Amazon couldn’t use all the resources they had available because there was a bug or some other issue with their software."
Although the outage lasted for hours on Prime Day, the impact on overall sales was minimal. Amazon said it was the “biggest shopping event” in company history, with over 100 million products purchased by Prime members during the 36-hour event. Half a dozen sellers who spoke to CNBC also said they were happy with this year’s Prime Day sales, even after dealing with the downtime.
Amazon hasn’t said much publicly about the outage. It issued a single statement two hours after the site crash, succinctly saying “some customers are having difficulty shopping” and that it was working to “resolve the issue quickly.”
In an internal email seen by CNBC, Jeff Wilke, Amazon’s CEO of worldwide retail, noted that his team was “disappointed” about the site issues and said the company’s already working on ways to prevent this from happening again. Then he highlighted all the ways that Prime Day was a success.
“Tech teams are already working to improve our architecture, and I’m confident we’ll deliver an even better experience next year,” he wrote in the email.
Amazon declined to comment.
The first hour
Amazon, based in Seattle, Washington, started seeing glitches across its site as soon as Prime Day launched at noon local time on Monday. In response, Amazon rushed to its backup plans and made quick changes during the first hour of the event.
Updates made at 12 p.m. say Amazon switched the front page to a simpler “fallback” page, as it saw a growing number of errors. Amazon’s front page on Prime Day looked oddly simple and rather poorly designed, noted Caesar, saying a simplified web page was likely put up to reduce load on their servers.
By 12:15 p.m., Amazon decided to temporarily cut off all international traffic to “reduce pressure” on its Sable system, and by 12:37 p.m., it reopened the default front page to only 25 percent of traffic. At 12:40 p.m., Amazon made certain changes that improved the performance of Sable, but just two minutes later, it went back to “consider” blocking approximately 5 percent of “unrecognized traffic to U.S.,” according to one of the documents.
Even after making these changes, Amazon’s site “error rate” continued to worsen until about 1:05 p.m., before drastically improving at 1:10 p.m., an internal site performance chart shows. Some parts of Amazon saw order rates that were “significantly higher than expected" by a factor of two, one of the updates said. One person familiar with the matter described the office scene as “chaotic” and said at one point more than 300 people tuned in to an emergency conference call.
“They are obviously scrambling, on short notice, to restore services,” said Henning Schulzrinne, a computer science professor at Columbia University and the former CTO of the Federal Communications Commission, after CNBC shared details of the documents. “These problems tend to feed on themselves — people retry loading, making the problem worse, or services complete partially. So shutting off services is often the better, but obviously bad, option.”
Internal system "Sable" on red alert
Amazon chose not to shut off its site. Instead, it manually added servers so it could improve the site performance gradually, according to the documents. One person wrote in a status update that he was adding 50 to 150 “hosts,” or virtual servers, because of the extra traffic.
Caesar says the root cause of the problem may have to do with a failure in Amazon’s auto-scaling feature, which automatically detects traffic fluctuations and adjusts server capacity accordingly. The fact that Amazon cut off international traffic first, rather than increase the number of servers immediately, and added server power manually instead of automatically, is an indication of a breakdown in auto-scaling, a critical component when dealing with unexpected traffic spikes, he said.
“If their auto-scaling was working, things would have scaled automatically and they wouldn't have had this level of outage,” Caesar said. “There was probably an implementation or configuration error in their automatic scaling systems.”
Due to the lack of server power, Amazon saw extra pressure on Sable, which is an internal storage and computational system that plays a critical role running multiple services across the site, according to documents seen by CNBC. Sable is used by 400 teams across Amazon and handled a total of 5.623 trillion service requests, or 63.5 million requests per second, during last year’s Prime Day, according to an internal document.
Sable was given a “red” emergency alert in one of the status updates, made a little past 1 p.m., which said it’s “running hot” and “cannot scale.” It also said other services, such as Prime, authentication and video playback, were being “impacted by Sable.”
“We are experiencing failures mostly related to Sable,” one of the updates said.
Carl Kesselman, a computer science professor at USC, said Amazon’s response to the outage was rather impressive because in many cases the site would have crashed entirely under those circumstances.
“Amazon is operating at a scale we haven’t operated before,” he said. “It’s not clear there’s a bad guy or an obvious screw-up. It’s just we’re in uncharted territory, and it’s amazing it didn’t just fall over.”
This year’s Prime Day was the first one run by Neil Lindsay, Amazon’s VP of worldwide marketing and Prime. Lindsay took over the Prime team after the former lead, Greg Greeley, left the company for Airbnb earlier this year.
https://www.google.com/amp/s/www.cn...-caused-prime-day-crash-company-scramble.html
Eugene Kim | @eugenekim222
Published 2:43 PM ET Thu, 19 July 2018Updated 6:16 PM ET Thu, 19 July 2018CNBC.com
- Amazon wasn't able to handle the traffic surge and failed to secure enough servers to meet the demand on Prime Day, according to expert review of internal documents obtained by CNBC.
- That led to a cascading series of failures, including a slowdown in its internal computation and storage service called Sable and other services that depend on it, including Prime, authentication and video playback.
- Amazon immediately launched a scaled-down "fallback" front page to reduce workload and temporarily killed all international traffic too.
Amazon.com founder and CEO Jeff Bezos
Amazon failed to secure enough servers to handle the traffic surge on Prime Day, causing it to launch a scaled-down backup front page and temporarily kill off all international traffic, according to internal Amazon documents obtained by CNBC.
And that took place within 15 minutes of the start of Prime Day — one of Amazon's biggest sales days every year.
The e-commerce giant also had to add servers manually to meet the traffic demand, indicating its auto-scaling feature may have failed to work properly leading up to the crash, according to external experts who reviewed the documents. “Currently out of capacity for scaling,” one of the updates said about the status of Amazon’s servers, roughly an hour after Prime Day’s launch. “Looking at scavenging hardware.”
A breakdown in an internal system called Sable, which Amazon uses to provide computation and storage services to its retail and digital businesses, caused a series of glitches across other services that depend on it, including Prime, authentication and video playback, the documents show.
Other teams, including Alexa, Prime Now and Twitch, also reported problems, while some warehouses said they weren’t even able to scan products or pack orders for a period of time.
The documents give a rare look into how Amazon responded to the higher-than-expected traffic surge on Prime Day, which caused glitchesacross the site for hours. It also illustrates the difficulty Amazon faced in dealing with the demand, despite its deep experience running a massive-scale website and one of the largest cloud computing platforms in the world.
“More people came in than Amazon could handle,” Matthew Caesar, a computer science professor at the University of Illinois and co-founder of cybersecurity firm Veriflow, said after CNBC shared the details of the documents. “And Amazon couldn’t use all the resources they had available because there was a bug or some other issue with their software."
Although the outage lasted for hours on Prime Day, the impact on overall sales was minimal. Amazon said it was the “biggest shopping event” in company history, with over 100 million products purchased by Prime members during the 36-hour event. Half a dozen sellers who spoke to CNBC also said they were happy with this year’s Prime Day sales, even after dealing with the downtime.
Amazon hasn’t said much publicly about the outage. It issued a single statement two hours after the site crash, succinctly saying “some customers are having difficulty shopping” and that it was working to “resolve the issue quickly.”
In an internal email seen by CNBC, Jeff Wilke, Amazon’s CEO of worldwide retail, noted that his team was “disappointed” about the site issues and said the company’s already working on ways to prevent this from happening again. Then he highlighted all the ways that Prime Day was a success.
“Tech teams are already working to improve our architecture, and I’m confident we’ll deliver an even better experience next year,” he wrote in the email.
Amazon declined to comment.
The first hour
Amazon, based in Seattle, Washington, started seeing glitches across its site as soon as Prime Day launched at noon local time on Monday. In response, Amazon rushed to its backup plans and made quick changes during the first hour of the event.
Updates made at 12 p.m. say Amazon switched the front page to a simpler “fallback” page, as it saw a growing number of errors. Amazon’s front page on Prime Day looked oddly simple and rather poorly designed, noted Caesar, saying a simplified web page was likely put up to reduce load on their servers.
By 12:15 p.m., Amazon decided to temporarily cut off all international traffic to “reduce pressure” on its Sable system, and by 12:37 p.m., it reopened the default front page to only 25 percent of traffic. At 12:40 p.m., Amazon made certain changes that improved the performance of Sable, but just two minutes later, it went back to “consider” blocking approximately 5 percent of “unrecognized traffic to U.S.,” according to one of the documents.
Even after making these changes, Amazon’s site “error rate” continued to worsen until about 1:05 p.m., before drastically improving at 1:10 p.m., an internal site performance chart shows. Some parts of Amazon saw order rates that were “significantly higher than expected" by a factor of two, one of the updates said. One person familiar with the matter described the office scene as “chaotic” and said at one point more than 300 people tuned in to an emergency conference call.
“They are obviously scrambling, on short notice, to restore services,” said Henning Schulzrinne, a computer science professor at Columbia University and the former CTO of the Federal Communications Commission, after CNBC shared details of the documents. “These problems tend to feed on themselves — people retry loading, making the problem worse, or services complete partially. So shutting off services is often the better, but obviously bad, option.”
Internal system "Sable" on red alert
Amazon chose not to shut off its site. Instead, it manually added servers so it could improve the site performance gradually, according to the documents. One person wrote in a status update that he was adding 50 to 150 “hosts,” or virtual servers, because of the extra traffic.
Caesar says the root cause of the problem may have to do with a failure in Amazon’s auto-scaling feature, which automatically detects traffic fluctuations and adjusts server capacity accordingly. The fact that Amazon cut off international traffic first, rather than increase the number of servers immediately, and added server power manually instead of automatically, is an indication of a breakdown in auto-scaling, a critical component when dealing with unexpected traffic spikes, he said.
“If their auto-scaling was working, things would have scaled automatically and they wouldn't have had this level of outage,” Caesar said. “There was probably an implementation or configuration error in their automatic scaling systems.”
Due to the lack of server power, Amazon saw extra pressure on Sable, which is an internal storage and computational system that plays a critical role running multiple services across the site, according to documents seen by CNBC. Sable is used by 400 teams across Amazon and handled a total of 5.623 trillion service requests, or 63.5 million requests per second, during last year’s Prime Day, according to an internal document.
Sable was given a “red” emergency alert in one of the status updates, made a little past 1 p.m., which said it’s “running hot” and “cannot scale.” It also said other services, such as Prime, authentication and video playback, were being “impacted by Sable.”
“We are experiencing failures mostly related to Sable,” one of the updates said.
Carl Kesselman, a computer science professor at USC, said Amazon’s response to the outage was rather impressive because in many cases the site would have crashed entirely under those circumstances.
“Amazon is operating at a scale we haven’t operated before,” he said. “It’s not clear there’s a bad guy or an obvious screw-up. It’s just we’re in uncharted territory, and it’s amazing it didn’t just fall over.”
This year’s Prime Day was the first one run by Neil Lindsay, Amazon’s VP of worldwide marketing and Prime. Lindsay took over the Prime team after the former lead, Greg Greeley, left the company for Airbnb earlier this year.
https://www.google.com/amp/s/www.cn...-caused-prime-day-crash-company-scramble.html
Internal documents show how Amazon scrambled to fix Prime Day glitches
Eugene Kim | @eugenekim222
Published 2:43 PM ET Thu, 19 July 2018Updated 6:16 PM ET Thu, 19 July 2018CNBC.com
Amazon.com founder and CEO Jeff Bezos
Amazon failed to secure enough servers to handle the traffic surge on Prime Day, causing it to launch a scaled-down backup front page and temporarily kill off all international traffic, according to internal Amazon documents obtained by CNBC.
And that took place within 15 minutes of the start of Prime Day — one of Amazon's biggest sales days every year.
The e-commerce giant also had to add servers manually to meet the traffic demand, indicating its auto-scaling feature may have failed to work properly leading up to the crash, according to external experts who reviewed the documents. “Currently out of capacity for scaling,” one of the updates said about the status of Amazon’s servers, roughly an hour after Prime Day’s launch. “Looking at scavenging hardware.”
A breakdown in an internal system called Sable, which Amazon uses to provide computation and storage services to its retail and digital businesses, caused a series of glitches across other services that depend on it, including Prime, authentication and video playback, the documents show.
Other teams, including Alexa, Prime Now and Twitch, also reported problems, while some warehouses said they weren’t even able to scan products or pack orders for a period of time.
The documents give a rare look into how Amazon responded to the higher-than-expected traffic surge on Prime Day, which caused glitchesacross the site for hours. It also illustrates the difficulty Amazon faced in dealing with the demand, despite its deep experience running a massive-scale website and one of the largest cloud computing platforms in the world.
“More people came in than Amazon could handle,” Matthew Caesar, a computer science professor at the University of Illinois and co-founder of cybersecurity firm Veriflow, said after CNBC shared the details of the documents. “And Amazon couldn’t use all the resources they had available because there was a bug or some other issue with their software."
Although the outage lasted for hours on Prime Day, the impact on overall sales was minimal. Amazon said it was the “biggest shopping event” in company history, with over 100 million products purchased by Prime members during the 36-hour event. Half a dozen sellers who spoke to CNBC also said they were happy with this year’s Prime Day sales, even after dealing with the downtime.
Amazon hasn’t said much publicly about the outage. It issued a single statement two hours after the site crash, succinctly saying “some customers are having difficulty shopping” and that it was working to “resolve the issue quickly.”
In an internal email seen by CNBC, Jeff Wilke, Amazon’s CEO of worldwide retail, noted that his team was “disappointed” about the site issues and said the company’s already working on ways to prevent this from happening again. Then he highlighted all the ways that Prime Day was a success.
“Tech teams are already working to improve our architecture, and I’m confident we’ll deliver an even better experience next year,” he wrote in the email.
Amazon declined to comment.
The first hour
Amazon, based in Seattle, Washington, started seeing glitches across its site as soon as Prime Day launched at noon local time on Monday. In response, Amazon rushed to its backup plans and made quick changes during the first hour of the event.
Updates made at 12 p.m. say Amazon switched the front page to a simpler “fallback” page, as it saw a growing number of errors. Amazon’s front page on Prime Day looked oddly simple and rather poorly designed, noted Caesar, saying a simplified web page was likely put up to reduce load on their servers.
By 12:15 p.m., Amazon decided to temporarily cut off all international traffic to “reduce pressure” on its Sable system, and by 12:37 p.m., it reopened the default front page to only 25 percent of traffic. At 12:40 p.m., Amazon made certain changes that improved the performance of Sable, but just two minutes later, it went back to “consider” blocking approximately 5 percent of “unrecognized traffic to U.S.,” according to one of the documents.
Even after making these changes, Amazon’s site “error rate” continued to worsen until about 1:05 p.m., before drastically improving at 1:10 p.m., an internal site performance chart shows. Some parts of Amazon saw order rates that were “significantly higher than expected" by a factor of two, one of the updates said. One person familiar with the matter described the office scene as “chaotic” and said at one point more than 300 people tuned in to an emergency conference call.
“They are obviously scrambling, on short notice, to restore services,” said Henning Schulzrinne, a computer science professor at Columbia University and the former CTO of the Federal Communications Commission, after CNBC shared details of the documents. “These problems tend to feed on themselves — people retry loading, making the problem worse, or services complete partially. So shutting off services is often the better, but obviously bad, option.”
Internal system "Sable" on red alert
Amazon chose not to shut off its site. Instead, it manually added servers so it could improve the site performance gradually, according to the documents. One person wrote in a status update that he was adding 50 to 150 “hosts,” or virtual servers, because of the extra traffic.
Caesar says the root cause of the problem may have to do with a failure in Amazon’s auto-scaling feature, which automatically detects traffic fluctuations and adjusts server capacity accordingly. The fact that Amazon cut off international traffic first, rather than increase the number of servers immediately, and added server power manually instead of automatically, is an indication of a breakdown in auto-scaling, a critical component when dealing with unexpected traffic spikes, he said.
“If their auto-scaling was working, things would have scaled automatically and they wouldn't have had this level of outage,” Caesar said. “There was probably an implementation or configuration error in their automatic scaling systems.”
Due to the lack of server power, Amazon saw extra pressure on Sable, which is an internal storage and computational system that plays a critical role running multiple services across the site, according to documents seen by CNBC. Sable is used by 400 teams across Amazon and handled a total of 5.623 trillion service requests, or 63.5 million requests per second, during last year’s Prime Day, according to an internal document.
Sable was given a “red” emergency alert in one of the status updates, made a little past 1 p.m., which said it’s “running hot” and “cannot scale.” It also said other services, such as Prime, authentication and video playback, were being “impacted by Sable.”
“We are experiencing failures mostly related to Sable,” one of the updates said.
Carl Kesselman, a computer science professor at USC, said Amazon’s response to the outage was rather impressive because in many cases the site would have crashed entirely under those circumstances.
“Amazon is operating at a scale we haven’t operated before,” he said. “It’s not clear there’s a bad guy or an obvious screw-up. It’s just we’re in uncharted territory, and it’s amazing it didn’t just fall over.”
This year’s Prime Day was the first one run by Neil Lindsay, Amazon’s VP of worldwide marketing and Prime. Lindsay took over the Prime team after the former lead, Greg Greeley, left the company for Airbnb earlier this year.
https://www.google.com/amp/s/www.cn...-caused-prime-day-crash-company-scramble.html
Eugene Kim | @eugenekim222
Published 2:43 PM ET Thu, 19 July 2018Updated 6:16 PM ET Thu, 19 July 2018CNBC.com
- Amazon wasn't able to handle the traffic surge and failed to secure enough servers to meet the demand on Prime Day, according to expert review of internal documents obtained by CNBC.
- That led to a cascading series of failures, including a slowdown in its internal computation and storage service called Sable and other services that depend on it, including Prime, authentication and video playback.
- Amazon immediately launched a scaled-down "fallback" front page to reduce workload and temporarily killed all international traffic too.
Amazon.com founder and CEO Jeff Bezos
Amazon failed to secure enough servers to handle the traffic surge on Prime Day, causing it to launch a scaled-down backup front page and temporarily kill off all international traffic, according to internal Amazon documents obtained by CNBC.
And that took place within 15 minutes of the start of Prime Day — one of Amazon's biggest sales days every year.
The e-commerce giant also had to add servers manually to meet the traffic demand, indicating its auto-scaling feature may have failed to work properly leading up to the crash, according to external experts who reviewed the documents. “Currently out of capacity for scaling,” one of the updates said about the status of Amazon’s servers, roughly an hour after Prime Day’s launch. “Looking at scavenging hardware.”
A breakdown in an internal system called Sable, which Amazon uses to provide computation and storage services to its retail and digital businesses, caused a series of glitches across other services that depend on it, including Prime, authentication and video playback, the documents show.
Other teams, including Alexa, Prime Now and Twitch, also reported problems, while some warehouses said they weren’t even able to scan products or pack orders for a period of time.
The documents give a rare look into how Amazon responded to the higher-than-expected traffic surge on Prime Day, which caused glitchesacross the site for hours. It also illustrates the difficulty Amazon faced in dealing with the demand, despite its deep experience running a massive-scale website and one of the largest cloud computing platforms in the world.
“More people came in than Amazon could handle,” Matthew Caesar, a computer science professor at the University of Illinois and co-founder of cybersecurity firm Veriflow, said after CNBC shared the details of the documents. “And Amazon couldn’t use all the resources they had available because there was a bug or some other issue with their software."
Although the outage lasted for hours on Prime Day, the impact on overall sales was minimal. Amazon said it was the “biggest shopping event” in company history, with over 100 million products purchased by Prime members during the 36-hour event. Half a dozen sellers who spoke to CNBC also said they were happy with this year’s Prime Day sales, even after dealing with the downtime.
Amazon hasn’t said much publicly about the outage. It issued a single statement two hours after the site crash, succinctly saying “some customers are having difficulty shopping” and that it was working to “resolve the issue quickly.”
In an internal email seen by CNBC, Jeff Wilke, Amazon’s CEO of worldwide retail, noted that his team was “disappointed” about the site issues and said the company’s already working on ways to prevent this from happening again. Then he highlighted all the ways that Prime Day was a success.
“Tech teams are already working to improve our architecture, and I’m confident we’ll deliver an even better experience next year,” he wrote in the email.
Amazon declined to comment.
The first hour
Amazon, based in Seattle, Washington, started seeing glitches across its site as soon as Prime Day launched at noon local time on Monday. In response, Amazon rushed to its backup plans and made quick changes during the first hour of the event.
Updates made at 12 p.m. say Amazon switched the front page to a simpler “fallback” page, as it saw a growing number of errors. Amazon’s front page on Prime Day looked oddly simple and rather poorly designed, noted Caesar, saying a simplified web page was likely put up to reduce load on their servers.
By 12:15 p.m., Amazon decided to temporarily cut off all international traffic to “reduce pressure” on its Sable system, and by 12:37 p.m., it reopened the default front page to only 25 percent of traffic. At 12:40 p.m., Amazon made certain changes that improved the performance of Sable, but just two minutes later, it went back to “consider” blocking approximately 5 percent of “unrecognized traffic to U.S.,” according to one of the documents.
Even after making these changes, Amazon’s site “error rate” continued to worsen until about 1:05 p.m., before drastically improving at 1:10 p.m., an internal site performance chart shows. Some parts of Amazon saw order rates that were “significantly higher than expected" by a factor of two, one of the updates said. One person familiar with the matter described the office scene as “chaotic” and said at one point more than 300 people tuned in to an emergency conference call.
“They are obviously scrambling, on short notice, to restore services,” said Henning Schulzrinne, a computer science professor at Columbia University and the former CTO of the Federal Communications Commission, after CNBC shared details of the documents. “These problems tend to feed on themselves — people retry loading, making the problem worse, or services complete partially. So shutting off services is often the better, but obviously bad, option.”
Internal system "Sable" on red alert
Amazon chose not to shut off its site. Instead, it manually added servers so it could improve the site performance gradually, according to the documents. One person wrote in a status update that he was adding 50 to 150 “hosts,” or virtual servers, because of the extra traffic.
Caesar says the root cause of the problem may have to do with a failure in Amazon’s auto-scaling feature, which automatically detects traffic fluctuations and adjusts server capacity accordingly. The fact that Amazon cut off international traffic first, rather than increase the number of servers immediately, and added server power manually instead of automatically, is an indication of a breakdown in auto-scaling, a critical component when dealing with unexpected traffic spikes, he said.
“If their auto-scaling was working, things would have scaled automatically and they wouldn't have had this level of outage,” Caesar said. “There was probably an implementation or configuration error in their automatic scaling systems.”
Due to the lack of server power, Amazon saw extra pressure on Sable, which is an internal storage and computational system that plays a critical role running multiple services across the site, according to documents seen by CNBC. Sable is used by 400 teams across Amazon and handled a total of 5.623 trillion service requests, or 63.5 million requests per second, during last year’s Prime Day, according to an internal document.
Sable was given a “red” emergency alert in one of the status updates, made a little past 1 p.m., which said it’s “running hot” and “cannot scale.” It also said other services, such as Prime, authentication and video playback, were being “impacted by Sable.”
“We are experiencing failures mostly related to Sable,” one of the updates said.
Carl Kesselman, a computer science professor at USC, said Amazon’s response to the outage was rather impressive because in many cases the site would have crashed entirely under those circumstances.
“Amazon is operating at a scale we haven’t operated before,” he said. “It’s not clear there’s a bad guy or an obvious screw-up. It’s just we’re in uncharted territory, and it’s amazing it didn’t just fall over.”
This year’s Prime Day was the first one run by Neil Lindsay, Amazon’s VP of worldwide marketing and Prime. Lindsay took over the Prime team after the former lead, Greg Greeley, left the company for Airbnb earlier this year.
https://www.google.com/amp/s/www.cn...-caused-prime-day-crash-company-scramble.html
Amazon Prime Day created a surge in health and safety complaints from exhausted workers
Complaints reportedly received from warehouse workers during Prime Day included stomach cramps, lack of access to water and time to visit the bathroom, sprains and musculoskeletal injuries
By STEPHEN ARMSTRONG
An Amazon fulfilment centre in Peterborough photographed in November 2017 in the run-up to the the company's annual Black Friday sale
CHRIS J RATCLIFFE/AFP/Getty Images
Amazon Prime Day broke records last week – with more than 100 million products sold – but proved the most controversial deal day to date with strikes breaking out across Europe and health and safety complaints from Amazon UK workers soaring by 209 per cent, according to workplace digital campaigning platform Organise.
“In the case of Prime Day we’re seeing the amplification of the stuff we get on a normal day turned up to 11,” says Usman Mohammed, lead campaigner at Organise, which compiles workplace complaints via email lists – soliciting views from employees at a number of UK companies. “The increased pace and increased targets means three time the packing for some people - even for slight workplace injuries, the rise in pain is exponential.”
Cannock Chase district council, the company’s prime authority which takes responsibility for every site across the UK.
In a statement, a council spokesperson confirmed that it had “received a number of complaints allegedly from Amazon employees which have been gathered by the online petition Organise”. The spokesperson said that the council would investigate any valid complaints, though added that it expected employees with a concern to take it up with their employer first.
Amazon says that targets haven’t been raised, water is openly available to all associates, breaks are longer than legally required with additional walking time included, associates have easy access to toilet facilities and the company does not monitor toilet breaks.
The company has 25,000 full time permanent employees in the UK, ranging from head office staff to warehouse workers. At a minimum, all full-time Amazon employees earn £8.35 an hour after their first two years. Employees, the Amazon spokesperson said, are offered private medical insurance, life assurance, and income protection, as well as a company pension plan.
How Amazon turned Prime Day into a dizzying shopping extravaganza
Amazon
How Amazon turned Prime Day into a dizzying shopping extravaganza
Launched the day after Amazon founder Jeff Bezos became the richest human being in modern history, according to the Bloomberg Billionaires Index, the 2018 edition of Prime Day saw warehouse workers in Germany, Poland and Spain take strike action in protests over pay, conditions and union recognition. In the UK, the GMB union applied to the UK’s Central Arbitration Committee for formal recognition for collective bargaining at Amazon’s Rugeley plant the week before Prime Day.
“There’s a lot of fear in the workplace about getting the sack for speaking to us so we’re arranging meetings in church halls and community centres to organise and fight for recognition,” says Mick Rix, the GMB’s national organising officer. “We’re even getting on the buses with workers and talking to them there. People talk about a race to the bottom – that race was over a long time ago. Workers in constant pain and afraid to complain.”
In May the GMB published Freedom of Information requests to ambulance services revealing ambulances had been called 600 times to Amazon warehouses in the past three years – including 115 call-outs to the company’s Rugeley warehouse, compared to eight call outs for a similar sized Tesco warehouse nearby. This autumn, according to Rix, the GMB will back “a series of personal injury claims in a coordinated legal action for warehouse workers – claiming Amazon’s working methods cause injuries and industrial diseases.”
“Ensuring the safety of associates is our number one priority,” Amazon’s spokesperson said. “While any serious incident is one too many, we learn and improve our programs working to prevent future incidents. We don’t recognise these allegations as an accurate portrayal of activities in our buildings.”
The debate over Amazon’s workplace conditions may be decided in court.
https://www.wired.co.uk/article/amazon-prime-day-uk-worker-health-and-safety
Complaints reportedly received from warehouse workers during Prime Day included stomach cramps, lack of access to water and time to visit the bathroom, sprains and musculoskeletal injuries
By STEPHEN ARMSTRONG
An Amazon fulfilment centre in Peterborough photographed in November 2017 in the run-up to the the company's annual Black Friday sale
CHRIS J RATCLIFFE/AFP/Getty Images
Amazon Prime Day broke records last week – with more than 100 million products sold – but proved the most controversial deal day to date with strikes breaking out across Europe and health and safety complaints from Amazon UK workers soaring by 209 per cent, according to workplace digital campaigning platform Organise.
“In the case of Prime Day we’re seeing the amplification of the stuff we get on a normal day turned up to 11,” says Usman Mohammed, lead campaigner at Organise, which compiles workplace complaints via email lists – soliciting views from employees at a number of UK companies. “The increased pace and increased targets means three time the packing for some people - even for slight workplace injuries, the rise in pain is exponential.”
Cannock Chase district council, the company’s prime authority which takes responsibility for every site across the UK.
In a statement, a council spokesperson confirmed that it had “received a number of complaints allegedly from Amazon employees which have been gathered by the online petition Organise”. The spokesperson said that the council would investigate any valid complaints, though added that it expected employees with a concern to take it up with their employer first.
Amazon says that targets haven’t been raised, water is openly available to all associates, breaks are longer than legally required with additional walking time included, associates have easy access to toilet facilities and the company does not monitor toilet breaks.
The company has 25,000 full time permanent employees in the UK, ranging from head office staff to warehouse workers. At a minimum, all full-time Amazon employees earn £8.35 an hour after their first two years. Employees, the Amazon spokesperson said, are offered private medical insurance, life assurance, and income protection, as well as a company pension plan.
How Amazon turned Prime Day into a dizzying shopping extravaganza
Amazon
How Amazon turned Prime Day into a dizzying shopping extravaganza
Launched the day after Amazon founder Jeff Bezos became the richest human being in modern history, according to the Bloomberg Billionaires Index, the 2018 edition of Prime Day saw warehouse workers in Germany, Poland and Spain take strike action in protests over pay, conditions and union recognition. In the UK, the GMB union applied to the UK’s Central Arbitration Committee for formal recognition for collective bargaining at Amazon’s Rugeley plant the week before Prime Day.
“There’s a lot of fear in the workplace about getting the sack for speaking to us so we’re arranging meetings in church halls and community centres to organise and fight for recognition,” says Mick Rix, the GMB’s national organising officer. “We’re even getting on the buses with workers and talking to them there. People talk about a race to the bottom – that race was over a long time ago. Workers in constant pain and afraid to complain.”
In May the GMB published Freedom of Information requests to ambulance services revealing ambulances had been called 600 times to Amazon warehouses in the past three years – including 115 call-outs to the company’s Rugeley warehouse, compared to eight call outs for a similar sized Tesco warehouse nearby. This autumn, according to Rix, the GMB will back “a series of personal injury claims in a coordinated legal action for warehouse workers – claiming Amazon’s working methods cause injuries and industrial diseases.”
“Ensuring the safety of associates is our number one priority,” Amazon’s spokesperson said. “While any serious incident is one too many, we learn and improve our programs working to prevent future incidents. We don’t recognise these allegations as an accurate portrayal of activities in our buildings.”
The debate over Amazon’s workplace conditions may be decided in court.
https://www.wired.co.uk/article/amazon-prime-day-uk-worker-health-and-safety
Google's Cloud CEO just took a jab at Amazon's Prime day failure
Google Cloud CEO Diane Greene delivers a keynote at the company's 2018 NEXT conference.
Google's Cloud CEO Diane Greene took a jab at Amazon for its recent Prime Day snafu while delivering her keynote at the company's NEXT conference in San Francisco.
The retail giant's major sale extravaganza, Prime Day, experienced all sorts of glitches last week, as its servers failed to handle the surge in traffic. Although Amazon said it posted record sales despite the site failures, the technology challenges inspired plenty of criticism directed toward Amazon Web Services.
Greene made a veiled reference to the embarrassing issue on Tuesday while talking to Mike McNamara, Target's chief information and digital officer, about why the retailer decided to move its cloud operations from AWS to Google Cloud Platform.
AWS is the market leader in cloud infrastructure, followed by Microsoft Azure and Google Cloud Platform.
https://www.google.com/amp/s/www.cn...e-takes-a-jab-at-amazons-prime-day-snafu.html
- Google Cloud CEO Diane Greene went after Amazon on stage at the company's NEXT conference.
- "We're proud of how we got you through Black Friday and Cyber Monday last fall," she told Target's chief digital officer.
Google Cloud CEO Diane Greene delivers a keynote at the company's 2018 NEXT conference.
Google's Cloud CEO Diane Greene took a jab at Amazon for its recent Prime Day snafu while delivering her keynote at the company's NEXT conference in San Francisco.
The retail giant's major sale extravaganza, Prime Day, experienced all sorts of glitches last week, as its servers failed to handle the surge in traffic. Although Amazon said it posted record sales despite the site failures, the technology challenges inspired plenty of criticism directed toward Amazon Web Services.
Greene made a veiled reference to the embarrassing issue on Tuesday while talking to Mike McNamara, Target's chief information and digital officer, about why the retailer decided to move its cloud operations from AWS to Google Cloud Platform.
AWS is the market leader in cloud infrastructure, followed by Microsoft Azure and Google Cloud Platform.
https://www.google.com/amp/s/www.cn...e-takes-a-jab-at-amazons-prime-day-snafu.html