Monday, April 06, 2015

Interview on early Amazon personalization and recommendations

Amazon.com in late 1996
Amazon.com in mid-1997
I have a long interview with the Internet History Podcast mostly about Amazon around 1997, especially the personalization, recommendation engine, and data-driven innovations at Amazon, and the motivation behind them.

I think the interview a lot of fun. It gives a view of what Amazon was like way back when it was just a bookstore only in the US, had just one webserver, and we barely could keep the website up with all the growth.

Lots of history of the early days of the web, well before CSS and Javascript, before cookies were even widely supported, and before scale out, experimentation and A/B testing, and large scale log analysis were commonplace.

Give the podcast a listen if you are interested in what the Web looked like back in 1997 and the motivation behind Amazon's personalization and recommendations.

Thursday, April 02, 2015

Quick links

What I've been thinking about lately:
  • "The chip is so low power that it can be powered off energy capture from the body ... 35 microamps of power per megahertz of processing ... and less than 200 nanoamps ... in deep sleep mode" ([1])

  • "Forgetting may be nearly as important as remembering in humans" ([1])

  • Only 40% of people use maps on their smartphone ([1] [2])

  • OkCupid and Dataclysm: "In the age of Big Data, the empirical has deciphered the intimate" ([1])

  • Cross functional teams might seem slower when you're in them, but, long-term, are more productive ([1])

  • Very good article on mostly evil uses of personalization ([1] [2])

  • "Fake accounts are given a veneer of humanity by copying profile information and photos from elsewhere ... [and] a picture of a beautiful woman" ([1])

  • "Because almost no one patches their BIOSes, almost every BIOS in the wild is affected by at least one vulnerability" ([1])

  • Cracking by forcing non-random memory errors, just about all RAM chips currently used are vulnerable ([1] [2] [3])

  • Computer security "backdoors will always turn around and bite you in the ass. They are never worth it." ([1] [2])

  • "Facts can only do so much. To avoid coming to undesirable conclusions, people can fly from the facts and use other tools in their deep belief protecting toolbox" ([1])

  • Why TV is losing viewers, the ads are annoying: "Decline caused by a migration of viewers from ad-supported platforms to non-ad-supported, or less-ad-supported platforms" ([1])

  • "The same dysfunctional folie a deux playing out between credulous tech media and even more credulous VC investors" ([1])

  • Does the difficulty of building intelligent systems grow exponentially as we make progress? That question has big implications for whether we should expect (or fear) an AI singularity. ([1])

  • Very fun version of Family Feud using Google search suggestions ([1] [2])

  • Do you know what you don't know? Try this confidence calibration quiz. ([1])

  • Love this quote: "I have thrown away a number of successful careers out of boredom" ([1])

  • Humor related to recommendation systems: "An exciting new system that takes all the bother, all the deciding, all the paying—all the shopping—out of shopping." ([1])

  • Two SMBC comics related to AI ([1] [2])

Saturday, March 07, 2015

Data Maven from Crunchzilla: A light introduction to statistics

Crunchzilla just launched Data Maven!

Data Maven from Crunchzilla is a light introduction to statistics and data analysis.

For too many teens and adults, if they think about statistics at all, they think it's boring, tedious, or too hard. Too many people have had the experience of trying to learn statistics, only to get bogged down in probability, theory, and math, without feeling that they were able to do anything with it.

Instead, your first exposure to statistics should be fun, interesting, and mostly easy. Data Maven from Crunchzilla is more of a game than a tutorial. To play, you answer questions and solve problems using real data. Statistics is your tool, and data provides your answers. At the end of Data Maven, you'll not only know a bit about statistics, but also maybe even start to think of statistics as fun!

Like programming, statistics and data analysis are tools that make you more powerful. If you know how to use these tools, you can do things and solve problems others cannot. Increasingly, across many fields, people who understand statistics and data analysis can know more, learn more, and discover more.

Data Maven is not a statistics textbook. It is not a statistics class. It is an introduction. Data Maven demystifies statistics. Teens and adults who try Data Maven build their intuition and spark their curiosity for statistics and data.

Please try Data Maven yourself! And please tell others you know who might enjoy it too!

Monday, March 02, 2015

More quick links

Some of the best of what I've been thinking about lately:
  • Great TED talk titled "The mathematics of love", but probably should be titled "A data analysis of love" ([1])

  • Manned submarines are about to become obsolete and be replaced by underwater drones ([1] [2] [3])

  • "No other algorithm scaled up like these nets ... It was a just a question of the amount of data and the amount of computations." ([1] [2])

  • What Google has done is a little like taking a person who's never heard a sound before, not to mention ever hearing language before, and trying to have them learn how to transcribe English speech ([1] [2])

  • Teaching a computer to achieve expert level play of old video games by mimicking some of the purpose of sleep ([1] [2])

  • "Computers are actually better at object recognition than humans now" ([1] [2] [3] [4])

  • The goal of Google Glass was a "remembrance agent" that acts as a second memory and gives helpful information recommendations in real time ([1] [2] [3])

  • A new trend, large VC investments in artificial intelligence ([1])

  • "Possibly the largest bank theft the world has seen" done using malware ([1])

  • "Users will prioritise immediate gain, and tend to dismiss consequences with no immediate visible effect" ([1] [2])

  • "Crowds can't be trusted". It's "really a game of spamfighting". ([1] [2])

  • SMBC comic: "All we have to do is build a trustworthiness rating system for all humans" ([1])

  • Dilbert describes most business books: "He has no idea why he succeeded" ([1])

  • Architect Clippy: "I see you have a poorly structured monolith. Would you like me to convert it into a poorly structured set of microservices?" ([1])

  • Man kicks robot dog. Watching the video, doesn't it make you feel like the man is being cruel? The motion of the robot struggling to regain its balance is so lifelike that it triggers an emotional response. ([1] [2] [3])

  • SMBC comic: "Are we ever going to use math in real life?" ([1])

Wednesday, February 04, 2015

Quick links

What has caught my attention lately:
  • "Ads are often annoying ... [and] the practice of running annoying ads can cost more money than it earns" ([1] [2] [3])

  • Robot plays beer pong, but the real story is the clever bean bag robotic gripper using the "jamming phase transition of granular materials" ([1] [2] [3])

  • Good list of features a modern phone should have but does not ([1])

  • "At this point, Apple is basically an iPhone company with a few other side businesses ... The iPhone accounted for ... a staggering 69 percent ... of Apple's revenue." ([1])

  • "We were not building the phone for the customer — we were building it for Jeff [Bezos]" ([1] [2])

  • "One of the biggest problems in organizations is that the meeting is a tool that is used to diffuse responsibility" ([1] [2])

  • Pew poll on how opinions of US scientists differ from the US population, and public's perceptions of scientists ([1])

  • Pair a "brash, young scientist" with a "wiser, older scientist" to maximize innovation ([1] [2] [3])

  • Google Earth Pro is now free, lets you get high res stills and movies of anywhere on the planet ([1] [2])

  • People told a placebo was "expensive" had twice the improvement as measured by physical tests and brain scans ([1])

  • Blind men successfully train themselves to "see" using echolocation, and brain scans determine that they are using the otherwise unused visual centers of their brains to do so ([1] [2] [3] [4] [5])

  • Rather than modeling crowds with attraction and repulsion between agents, only avoiding anticipated collisions behaves closer to real humans ([1])

  • Xkcd comic: "I can't wait for the day when all my stupid computer knowledge is obsolete" ([1])

  • Xkcd What If: "Getting to space is easy. The problem is staying there." ([1])

Wednesday, January 14, 2015

More on what to advertise when there is no commercial intent

Some of the advertising out there is getting spooky. If you look at a product at many online stores, that product will then follow you around the web.

Go to BBC News, for example, and there will be those dishes you were looking at yesterday on Overstock. Not just any dishes, the exact same dishes. Just in case you forgot about them, there they are again next time you go. And again. And again.

A few years ago, I wrote an article, "What to advertise when there is no commercial intent?". That article suggested that, on sites like news sites, we might not have immediate commercial intent, and might have to reach back into the past to find strong commercial intent. It advocated for personalized advertising that helped people discover interesting products and deals related to strong commercial intent they had earlier.

However, this did not mean that you should just show the last product I looked at. That is refinding, not personalized recommendations. Refinding is all a lot of these ads are doing. You look at a chair, ads follow you around the web showing you ads for that same chair that you already know about over and over again. That's not discovery. That's spooky and not helpful.

Personalized ads should help people discover things they don't know about related to past purchase intent. If I look at a chair, show me highly reviewed similar furniture and good coupons and big deals related in some non-obvious way to that chair and that store. Don't just show me the same chair again. I know about that chair. Show me something I don't know. Help me discover something I haven't found yet.

I understand the reason these companies are doing refinding is because it's hard to do anything better. Doing useful recommendations of related products and deals is hard. Helping people discover something new and interesting is hard. Personalized recommendations requires a lot of data, clever algorithms, and a huge amount of work. Refinding is trivially easy.

But publishers aren't doing themselves any favors by allowing these startups to get away with this kind of useless advertising. As a recent study says, "the practice of running annoying ads can cost more money than it earns." That short-term revenue bump from these spooky refinding ads is like a sugar rush, feels good while it lasts, but hurts in the long-term.

They can and should do better. Personalization, including personalized advertising, should be about helping people discover things they could not easily find on their own. Personalization should not be refinding, just showing what I found before, just exposing my history. Personalization should be helpful. Personalization should be discovery.

Friday, January 02, 2015

Quick links

Some of the best of what I've been thinking about lately:
  • Tiny cheap satellites will provide near real-time imagery of the entire Earth to anyone who wants it, starting in about a year ([1] [2] [3])

  • Amplifying motion and color changes in video, which allows augmented perception ([1] [2])

  • Birds can hear the very low frequency sound produced by severe weather and are able to flee well in advance of incoming storms ([1])

  • Nice example of blending computer science with another field, in this case genealogy, to yield big new gains ([1])

  • "An energy gradient 1000 times greater than traditional particle accelerators" ([1])

  • People "don't want to watch commercials, are fleeing networks, hate reruns, are increasingly bored by reality programming, shun print products and, oh, by the way, don’t want to pay much for content either. Yikes." ([1] [2])

  • Everything we know Google is working on ([1])

  • Funny and informative: "Riding in a Google Self-Driving Car" ([1])

  • Google is rejecting security based on firewalls ([1] [2] [3])

  • "Whether you call it a Star Trek Universal Translator or Babel fish, Microsoft is building it, and it's incredible." ([1])

  • "Every dollar a worker earns in a research field spills over to make the economy $5 better off. Every dollar a similar worker earns in finance comes with a drain, making the economy 60 cents worse off." ([1])

  • "I’m a big believer in making effectively infinite computing resources available internally ... [Give] teams the resources they need to experiment ... All employees should be limited only by their ability rather than an absence of resources or an inability to argue convincingly for more." ([1])

  • "We think of it as a one-on-one tutor. It will test you and generate a personal lesson plan just for you." ([1])

  • "Apparently, a sufficient number of puppies can explain any computer science concept. Here we have multithreading:" ([1])

  • Fantastic to see a US president promoting computer programming to kids: "Becoming a computer scientist isn't as scary as it sounds. With hard work and a little math and science, anyone can do it." ([1])

Wednesday, December 03, 2014

More quick links

More of what caught my attention lately:
  • "Make infinite computing resources available internally ... Give teams the resources they need to experiment ... All employees should be limited only by their ability rather than an absence of resources or an inability to argue convincingly for more." ([1] [2])

  • "Accept that failures will always happen and guard ... [against] cascading failures by purposefully causing failures" ([1] [2])

  • "The importance of Netflix’s recommendation engine is actually underestimated" ([1] [2])

  • Courts are getting more skeptical about software patents ([1])

  • Nice way of putting it: "The prevailing business culture in the banking industry weakens and undermines the honesty norm" ([1] [2])

  • "[On] the overcrowded, overstuffed, slow-loading web, you are bound to see a carnival of pop-ups and interstitials — interim ad pages served up before or after your desired content — and scammy come-ons daring you to click. Is it any wonder, really, that this place is dying?" ([1])

  • A very effective social engineering attack "compromised the accounts of C-level executives, legal counsel, regulatory and compliance personnel, scientists, and advisors of more than 100 [major] companies" ([1])

  • An 11 hour Microsoft Azure cloud service outage that impacted just about everyone using it worldwide, including internal users like MSN.com and Xbox Live ([1])

  • Stack traces at arbitrary break points in Google's cloud services running live with near zero overhead ([1] [2])

  • Free SSL certificates (for HTTPS) from a non-profit out of EFF, Mozilla, Cisco, and Akamai ([1])

  • The journal Nature makes its papers free for everyone to read ([1] [2])

  • Combining neural networks like components yields new breakthroughs ([1] [2])

  • Robotics guru Rodney Brooks says, "Relax. Chill ... [The press has a] misunderstanding of how far we really are from having volitional or intentional artificially intelligent beings." ([1])

  • Undersea drones are enabling new feats: "The first time ... the black sea devil anglerfish ... has been filmed alive and in its natural habitat" ([1])

  • Bats jam the sonor of other bats when they're both trying to catch the same insect. It's like a dogfight up there. ([1])

  • Great tutorial on CSS and HTML just launched by Khan Academy and jQuery's John Resig ([1])

  • Fun visualization of the periodic table by how common the elements are in the earth's crust, ocean, human body, and sun ([1])

  • Hilarious parody of the Amazon Echo promotional video ([1])

  • South Park has a surprisingly good (and funny) criticism of freemium games that gets all the issues correct around preying on people with a tendency toward compulsive gambling ([1] [2])

  • Great Dilbert comic on how engineers think of marketing ([1])

  • Good Xkcd comic on over-optimization ([1])

  • Loved this SMBC comic: "He said I wasn't very good at math" ([1]) 

Monday, November 03, 2014

Quick links

What has caught my attention recently:
  • Netflix says the value of its recommendations algorithms is $500M/year ([1])

  • Details on the internals of LinkedIn's recommender system ([1])

  • Fantastic list of some hard and interesting big data problems at Facebook ([1] [2])

  • Google Glass may target "'superhero vision', like seeing in the dark, or magnifying subtle motion or changes" ([1] [2])

  • A claim that Amazon's cloud revenue is $4.7B this year, supposedly x30 bigger than Microsoft ($156M) and x70 Google's ($66M) ([1])

  • "We have a 10 petabyte data warehouse on S3" ([1])

  • Google's Eric Schmidt says, "Our biggest search competitor is Amazon" ([1])

  • Apple was and still is almost entirely an iPhone company ([1])

  • Tablet sales are projected to be flat now, and the growth boom for tablets appears to be done ([1])

  • But, it's interesting that specialized, expensive, and often poorly done custom hardware is getting replaced with a cheap touchscreen tablet ([1])

  • So far, it doesn't look like Windows 10 is going to fix what was wrong with Windows 8 ([1])

  • What? "Microsoft loves Linux" ([1] [2])

  • Delivery startups are back: "Silicon Valley wants to save you from ever having to leave your couch. Will it work this time around?" ([1])

  • Despite the difficulty older adults have with tiny mobile keyboards, older adults and seniors don't use voice search much ([1])

  • Speculation that hardware to enable gesture control on mobile phones will be widespread on new phones next year ([1])

  • A claim that "solar will soon reach price parity with conventional electricity in well over half the nation: 36 states" ([1])

  • "HP’s Multi Jet Fusion printer can crank out objects 10 times faster than any machine that’s on the market today ... 3D print heads that can operate 10,000 nozzles at once, while tracking designs to a five-micron precision." ([1] [2])

  • Is biology about to be transformed by the use of many drones to gather lots of data? ([1] [2])

  • More evidence that some of the best innovations come from combining ideas from two very separate fields ([1])

  • "Every success in AI redefines it. But we haven't just been redefining what we mean by AI-we've been redefining what it means to be human [and intelligent]." ([1])

  • "China is merely regaining a title that it has held for much of recorded history" ([1])

  • Funny Dilbert comic on multitasking and checking e-mail too often ([1])

  • The Onion: "This already vanishing glimmer of pleasure is exactly what we've come to expect from Apple" ([1])

  • Great SMBC comic: "The humans aren't doing what the math says. The humans must be broken." ([1])

Saturday, October 25, 2014

At what point is an over-the-air TV antenna too long to be legal?

You can get over-the-air HDTV signals using an antenna. This antenna gets a better, stronger signal with less interference if it is direct line-of-sight and as near as possible to the broadcast towers. So, you might want an antenna that is up high or even some distance away to get the best signal.

But if you try to do this, you immediately run into a question: At what point does that antenna become too long to be legal or the signal from the antenna is transmitted in a way where it is no longer legal?

Let's say I put an antenna behind my TV hooked up with a wire. That's obviously legal and what many people currently do.

Let's say I put an antenna outside on top of a tree or my garage and run a wire inside. Still seems obviously legal.

Let's say I put an antenna on top of my roof. Still clearly fine.

Let's say I put it on my neighbor's roof and run a wire to my TV. Still ok?

Let's say I put the antenna on my neighbor's roof, but have the antenna connect to my WiFi network and transmit the signal using my local area network instead of using a direct wired cable connection. Still ok?

Let's say I put the antenna on my neighbor's roof, but have the antenna connect to my neighbor's WiFi network and transmit the signal over their WiFi, over the internet, then to my WiFi, instead of using a direct wired cable connection. Still ok?

Let's say I put my antenna on my neighbor's roof, but my neighbor won't do this for free. I have to pay a small amount of rent to my neighbor for the space on his roof used by my antenna. I also have the antenna connect to my neighbor's WiFi network and transmit its signal over their WiFi, over the internet, then to my WiFi, instead of using a direct wired cable connection. Still ok?

Let's say, like before, I put my antenna on my neighbor's roof, pay the neighbor rent for the space on his roof, use the internet to transmit the antenna's signal. But, this time, I buy the antenna from my neighbor at the beginning (and, like before, I own it now). Is that okay?

Let's say I put my antenna on my neighbor's roof, pay the neighbor rent for the space on his roof, use the internet to transmit the antenna's signal, but now I rent or lease the antenna from my neighbor. Still ok? If this is not ok, which part is not ok? Is it suddenly ok if I replace the internet connection with a direct microwave relay or hardwired connection?

Let's say I do all of the last one, but use a neighbor's roof three houses away. Still ok?

Let's say I do all of the last one, but use a roof on a building five blocks away. Still ok?

Let's say I rent an antenna on top of a skyscraper in downtown Seattle and have the signal sent to me over the internet. Not ok?

The Supreme Court recently ruled Aereo is illegal. Aereo put small antennas in a building and rented them to people. The only thing they did beyond the last thing above is time-shifting, so they would not necessary send the signal from the antenna immediately, but instead store it, and only transmit it when demanded.

You might think it's the time shifting that's the problem, but that didn't seem to be what the Supreme Court said. Rather, they said the intent of the 1976 amendments to US copyright law prohibit community antennas (which is one antenna that sends its signal to multiple homes), labelling those a "public performance". They said Aereo's system was similar in function to a community antenna, despite actually having multiple antennas, and violated the intent of the 1976 law.

So, the question is, where is the line? Where does my antenna become too distant, transmit using the wrong methods, or involve too many payments to third parties in the operation of the antenna that it becomes illegal? Can it not be longer than X meters? Not transmit its signal in particular ways? Not require rent for the equipment or space on which the antenna sits? Not store the signal at the antenna and transmit it only on demand? What is the line?

I think this question is interesting for two reasons. First, as an individual, I would love to have a personal-use over-the-air HDTV antenna that gets a much better reception than the obstructed and inefficient placement behind my TV, but I don't know at what point it becomes illegal for me to place an antenna far away from the TV. Second, I suspect many others would like a better signal from their HDTV antenna too, and I'd love to see a startup (or any group) that helped people set up these antennas, but it is very unclear what it might be legal for a startup to do.

Thoughts?

Friday, October 24, 2014

Why can't I buy a solar panel somewhere else in the US and get a credit for the electricity from it?

Seattle City Light has a clever project where, instead of installing solar panels on your house where they might be obscured by trees or buildings, you can buy into a solar panel installation on top of a building in a more efficient location and get a credit for the electricity generated on your electric bill.

Why stop there? Why can't I buy a solar panel in a very different location and get the electricity from it?

Phoenix, Arizona has about twice the solar energy efficiency of Seattle. Why can't I buy a solar panel and enjoy the electricity credit from that solar panel when it is installed in a nice sunny spot in the Southwest?

This doesn't require shipping the actual electricity to your home. Instead, you fund an installation of solar panels on top of a building in an area of the US with high solar energy efficiency, then get a credit for that electricity on your monthly electricity bill.

I suppose, at some boring financing level, this starts to resemble a corporate bond, with an initial payment yielding a stream of payments over time, but people wouldn't see it that way. The attraction would be installing solar panels and getting a credit on your energy bill without installing solar panels on your own home. Perhaps the firm arranging the installations and working out the deals with local utilities could be treating the entire thing as the equivalent of marketing bonds to people who like solar energy, but the attraction to people is that visceral appeal of a near $0 electricity bill they see every month from the solar panels they feel like they own and installed.

Even with the overhead pulled out by the company selling this and arranging deals with local utilities so this all appears on your local electricity bill, the credit on your electricity bill still should be much higher than you could possibly get installing panels on your own home with all its obstructions and cloudy weather. Solar generation in an ideal location in the US easily can generate twice as much power as what is available locally, on your rooftop.

So, why hasn't someone done this? Why can't I buy solar panels and have them installed not on my own home, but in some much better spot?

Wednesday, October 01, 2014

Quick links

What caught my attention lately:
  • 12% of Harvard is enrolled in CS 50: "In pretty much every area of study, computational methods and computational thinking are going to be important to the future" ([1])

  • Excellent "What If?" nicely shows the value of back-of-the-envelope calculations and re-thinking what exactly it is you want to do ([1])

  • The US has almost no competition, only local monopolies, for high speed internet ([1] [2])

  • You can't take two large, dysfunctional, underperforming organizations, mash them together, and somehow make diamonds. When you take two big messes and put them together, you just get a bigger mess. ([1])

  • "Yahoo was started nearly 20 years ago as a directory of websites ... At the end of 2014, we will retire the Yahoo Directory." ([1] [2])

  • Investors think that Yahoo is essentially worthless ([1])

  • "At a moment when excitement about the future of robotics seems to have reached an all-time high (just ask Google and Amazon), Microsoft has given up on robots" ([1])

  • "Firing a bunch of tremendously smart and creative people seems misguided. But hey—at least they own Minecraft!" ([1])

  • "Macs still work basically the same way they did a decade ago, but iPhones and iPads have an interface that's specifically designed for multi-touch screens" ([1] [2])

  • On the difficulty of doing startups ([1] [2])

  • "Be glad some other sucker is fueling the venture capital fire" ([1])

  • "Just how antiquated the U.S. payments system has become" ([1])

  • Is everyone grabbing money from online donations to charities? Visa's charge fee on charities is only 1.35%, but the lowest online payment system for charities charges 2.2% and most charge much more than that. ([1])

  • "For most people, the risk of data loss is greater than the risk of data theft" ([1])

  • Password recovery "security questions should go away altogether. They're so dangerous that many security experts recommend filling in random gibberish instead of real answers" ([1])

  • Brilliantly done, free, open source, web-based puzzle game with wonderfully dark humor about ubiquitous surveillance ([1])

  • How Udacity does those cool transparent hands in its videos ([1])

  • There's just a bit of interference when you move your hand above the phone, just enough interference to detect gestures without using any additional power or sensors ([1] [2])

  • Small, low power wireless devices powered by very small fluctuations in temperature ([1] [2])

  • Cute intuitive interface for transferring data between PC and mobile ([1] [2])

  • "Federal funding for biomedical research [down 20%] ... forcing some people out of science altogether" ([1])

  • Another fun example of virtual tourism ([1])

  • Ig Nobel Prizes: "Dogs prefer to align themselves to the Earth's north-south magnetic field while urinating and defecating" ([1])

  • Xkcd: "In CS, it can be hard to explain the difference between the easy and the virtually impossible" ([1] [2])

  • Dilbert: "That process sounds like a steaming pile of stupidity that will beat itself to death in a few years" ([1])

  • Dilbert on one way to do job interviews ([1])

  • The Onion: "Startup Very Casual About Dress Code, Benefits" ([1])

  • Hilarious South Park episode, "Go Fund Yourself", makes fun of startups ([1])

Monday, September 08, 2014

The problem with personalized education

Personalized education has had some spectacular failures lately, in large part due to how tone-deaf the backers have been to the needs of teachers, parents, and students.

The right way to do personalization is to prove you're useful first. Personalization is just a tool. If a new tool doesn't work better than the old tool, it's useless. There's no reason to use personalized education unless it works better than unpersonalized education. A tool needs to be useful.

Teachers are already overworked and, after having been burned too many times on supposedly exciting new technologies that fail to help, correctly are cynical about tech startups coming in and demanding something of them. If some tech startup isn't helping a teacher get something done they need to get done, it's a bad tool and it's useless.

Parents are leery of companies who say they only want to help and what corporations are doing with the data they have on their children, correctly so given all the marketing abuses that have happened in the past.

Kids don't want more boring busywork to do -- they get enough of that already -- and don't see why anything this company is talking about helps them or is useful to them.

If a company wants to succeed in personalized education, it should:
  1. Be useful, noticeably raise test scores
  2. Not require additional busy work
  3. Be optional
  4. Have no marketing whatsoever, only use data to help
I think there are plenty of examples of how this might work. I would like to see a company offer a free Duolingo-like pre-algebra and algebra app that jumps students ahead rapidly as they answer questions correctly and spends more time on similar problems after a question is wrong. The app would be completely optional for students to use, but, when students use it, their test scores increase.

I would like to see a company use the existing standardized tests required by several states, analyze the incorrect answers to identify concepts a student is not understanding, and then print short worksheets targeting only those missed concepts for teachers to hand out to each student. The worksheets would be free and arrive in teachers' mailboxes. If the teacher doesn't want to hand them out, that's not a problem, but test scores go up for the classrooms where the teachers do hand them out. So, even if most teachers don't hand them out at first and most students throw them away at first, over time, more and more teachers will start handing them out and more and more students will do them, as only helps those who do.

In both of these examples, a startup could set up from the beginning to run large scale experiments, showing different problems to different students, and learning what raises test scores, what designs and lesson lengths cause students to stop, what concepts are important and which matter less, what can be taught easily through this and what cannot, what people enjoy, and what works.

When a company comes in and says, "Give us your data, teachers, parents, and kids, and do all this work. Maybe we'll boost your test scores for you later," they're being arrogant and tone-deaf. Everyone responds, "I don't believe you. How about you prove you're useful first? I'm busy. Do something for me or go away." And they're right to do so.

There likely is a way to do personalized education that everyone would embrace. But that way probably requires proving you're useful first. After all, personalization is just a tool.

Wednesday, September 03, 2014

More quick links

More of what caught my attention lately:
  • The overwhelming majority of smartphone users set up their phone once, then barely ever download a new app again ([1] [2])

  • Cool and successful use of speculative execution in cloud computing for games, trading off extra CPU and bandwidth for the ability to hide network latency ([1])

  • Infrared vision on your phone ([1] [2])

  • How easy is it to get people to memorize hard-to-crack random 56-bit passwords, equivalent to about 12 random letters or 6 words? ([1] [2])

  • Desalination needs warm water, data centers need to be cooled, why not put them together? Clever idea. ([1])

  • It's easy to overhype this, but it's still pretty cool, transmitting data (0 and 1 bits) directly brain-to-brain without implants (using magnetic stimulation of the brain and EEG reading of the brain, both from the surface of the scalp) with relatively low error rates (5-15%). Data rates are extremely low at 2-3 bits/minute, but it's still interesting that it's possible at all. ([1])

  • Xiaomi's remarkable iPhone clone ([1])

  • Has Amazon sold less than 35k Fire phones? ([1] [2])

  • Facebook publishes a paper which details how its ad targeting works and suggests they will be doing more personalization in the future ([1] [2])

  • "Having a multiyear project with no checks along the way and the promise of one big outcome is not a highly successful approach, in or outside government" ([1] [2])

  • More evidence patent trolls cause real harm. Trolled firms "dramatically reduce R&D spending". ([1])

  • "Using nothing more than a laptop ... [they could] alter the normal timing pattern of the [traffic] lights, turning all the lights along a given route green, for instance, or freezing an intersection with all reds" ([1])

  • Interesting data visualization showing how CD took over in music sales, then got replaced by downloads, all over the last two decades or so ([1])

  • Neat charts on how the strike zone expands on 3 ball counts and contracts on 2 strike counts ([1])

  • Cute SMBC comic on "What is the fastest animal?" ([1])

  • Great SMBC comic on job interviews ([1])

Tuesday, August 05, 2014

Quick links

What caught my attention lately:
  • Great idea for walking directions: "At times, we do not [want] the fastest route ... When walking, we generally prefer tiny streets with trees over large avenues with cars ... [We] suggest routes that are not only short but also emotionally pleasant." ([1] [2] [3])

  • Cool idea for a drone that autonomously flies a small distance above and behind you while filming in HD ([1] [2])

  • "OkCupid doesn’t really know what it’s doing. Neither does any other website. It’s not like people have been building these things for very long, or you can go look up a blueprint or something. Most ideas are bad. Even good ideas could be better. Experiments are how you sort all this out." ([1] [2])

  • "Amazon’s cloud revenue now runs almost on par with VMware (VMW), which posted revenue of $5.2 billion last year" ([1])

  • Walmart is getting more aggressive about competing with Amazon on personalization and recommendations ([1])

  • It's important to realize that Amazon could have been a small bookstore on the Web ([1])

  • A lot of us thought the Amazon logo was phallic when it was introduced (worse, it was animated and actually grew from left-to-right). Remarkably, it's lived on for 14 years now. ([1])

  • A big problem with layoffs is not only do you lose some of the people you intended to layoff, but also some of your best employees will pick that time to leave. People with good options won't wait around to experience the chaos and fear; they'll just leave. ([1])

  • "A brand-name USB stick [claims to be] a computer keyboard [device] ... [and then] opens a command window on an attached computer and enters commands that cause it to download and install malicious software." ([1])

  • Financial services and poor computer security: "Our assumption was that, generally speaking, the financial sector had its act together much more" ([1] [2])

  • "NSA employees [were] passing around nude photos that were intercepted in the course of their daily work" ([1] [2])

  • Google Cloud googler says, "It should always be cheaper to run in the cloud no matter what your workload" but that the pricing isn't there yet ([1])

  • Details on Google's remarkably large and fast data warehouse ([1] [2])

  • Cool augmented reality game intended to be played as a passenger in a moving car that creates the terrain and enemies you see in the game based on the stores and buildings around you in the real world ([1])

  • "Astronomers of the 2020s will be swimming in petabytes of data streaming from space and the ground ... [such as] a 3,200-megapixel camera, which will produce an image of the entire sky every few days and over 10 years will produce a movie of the universe, swamping astronomers with data that will enable them to spot everything that moves or blinks in the heavens, including asteroids and supernova explosions." ([1])

  • Data are or data is: "'datum' isn't a word we ever use. So it makes no sense to use the plural when the singular doesn't exist." ([1])

  • The "If Google was a guy" series from CollegeHumor is hilarious (but probably NSFW) ([1] [2] [3])

  • Funny Dilbert comics on a Turing test for management ([1] [2])

  • Cathartic Xkcd comic on defending your thesis ([1])

Wednesday, July 02, 2014

More quick links

More of what caught my attention lately:
  • Crazy cool and the first time I've seen ultrasound used for device-to-device communication outside of research: "Chromecast will be able to pair without Wi-Fi, or even Bluetooth, via an unusual method: ultrasonic tones." ([1])

  • A 3D printer that can print in "any weldable material" including titanium, aluminum, and stainless steel ([1])

  • "You teach Baxter [an inexpensive industrial robot] how to do something by grabbing an arm and showing it what you want, sort of like how you would teach a child to paint" ([1])

  • When trying to use the wisdom of the crowds, you're better off using only the best part of the crowd. ([1])

  • "Americans now appear to trust internet news about as much as newspapers and television news ... not because confidence in internet news is rising, but because confidence in TV news and newspapers has plummeted over the years." ([1])

  • "Microsoft is basically 'done' with Windows 8.x. Regardless of how usable or functional it is or isn't, it has become Microsoft's Vista 2.0 -- something from which Microsoft needs to distance itself." ([1])

  • Google Flights now lets you see everywhere you can fly out of a city (including limiting to non-stops only) and how much it would cost ([1] [2] [3] [4])

  • "Entering the fulfillment center in Phoenix feels like venturing into a realm where the machines, not the humans, are in charge ... The place radiates a non-human intelligence, an overarching brain dictating the most minute movements of everyone within its reach." ([1])

  • Google's location history feature is both fascinating and frightening. If you own an Android device, go to location history, set it to 30 days, and see the detail on where you have been. While it's true that many have this kind of data, it may surprise you to see it all at once.

  • "Vodafone, one of the world's largest mobile phone groups, has revealed the existence of secret wires that allow government agencies to listen to all conversations on its networks, saying they are widely used in some of the 29 countries in which it operates in Europe and beyond." ([1])

  • Many "users actually do not attach any signi´Čücant economic value to the security of their systems" ([1] [2])

  • "Ensuring that our patent system 'promotes the progress of science,' rather than impedes it, consistent with the constitutional mandate underlying our intellectual property system" ([1])

  • Smartphones may have hit the limit on how much improvements to screen resolution matter, meaning they will have to compete on other features (like sensors or voice recognition) ([1])

  • "Project Tango can see the world around it in 3D. This would allow developers to make augmented-reality apps that line up perfectly with the real world or make an app that can 3D scan an object or environment." ([1])

  • The selling point of smartwatches is paying $200 to not have to pull your phone out of your pocket, and that might be a tough sell. ([1])

  • "As programmers will tell you, the building part is often not the hardest part: It's figuring out what to build. 'Unless you can think about the ways computers can solve problems, you can't even know how to ask the questions that need to be answered'" ([1])

  • "[No] lectures, discussion sections, midterms ... a pre-test for each subject area ... given a mentor with a graduate degree in the field ... [and] textbooks, tutorials, and other resources. Eventually, they're assessed on how well they understand the concepts." ([1])

  • "A naked mole rat has never once been observed to develop cancer" ([1])

  • Hilarious Colbert Report on the Hachette mess, particularly loved the bit on "Customers who enjoyed this also bought this" at 3:00 in the video ([1])

  • Humor from The Onion: "We want $100 from you, so we’re just going to take it. As a cable subscriber, you really have no other option here" ([1])

  • Humor from the Borowitz Report: "It never would have occurred to me that an enormous corporation with the ability to track over half a billion customers would ever exploit that advantage in any way." ([1])

Thursday, June 05, 2014

Quick links

What caught my attention lately:
  • Fun data: "How to tell someone's age when all you know is her name" ([1])

  • "The possibility of proper tricorder technology in the future, scanning a bit of someone's blood and telling you if they have any diseases or anomalous genetic conditions" ([1])

  • Will self-driving vehicles appear first in trucking? ([1])

  • "Apple's moves into the world of fashion and wearable computing" ([1])

  • "Few people try to or want to use tablets like laptops" ([1] [2])

  • "While managers do indeed add value to a company, there’s no particular reason to believe that they add more value to a company than the people who report to them ... [You want] an organization where fairly-compensated people work together as a team, rather than trying to work out the best way to make money for themselves at the expense of their colleagues." ([1])

  • "Each meeting ... spawns even more meetings ... The solution ... reduce default meeting length from 60 to 30 minutes ... limit meetings to seven or fewer participants ... agendas with clear objectives ... materials ... distributed in advance .. on-time start ... early ending, especially if the meeting is going nowhere ... remove ... unnecessary supervisors." ([1])

  • Fun article on the history of the modern office: "The cubicle was actually intended to be this liberating design, and it basically became perverted" ([1])

  • "We were wrong about the first-time shoppers. They did mind registering. They resented having to register when they encountered the page. As one shopper told us, 'I'm not here to enter into a relationship. I just want to buy something.'" ([1])

  • Private investment in broadband infrastructure is actually dropping in the US ([1])

  • "Not only are packets being dropped, but all those not being dropped are also subject to delay. ... They are deliberately harming the service they deliver to their paying customers ... Shouldn't a broadband consumer network with near monopoly control over their customers be expected, if not obligated, to deliver a better experience than this?" ([1])

  • Fascinating data on cancer shows a surprising lack of linear relationship between aging and cancer ([1] [2])

  • "A wayward spacecraft ISEE-3/ICE was returning to fly past Earth after many decades of wandering through space. It was still operational, and could potentially be sent on a new mission, but NASA no longer had the equipment to talk to it ... crowdfunding project ... commandeer the spacecraft ... awfully long shot ... They are now in command of the ISEE-3 spacecraft." ([1])

  • I love the caption on this comic: "Somebody please do this and post it on YouTube so I can live vicariously through your awesomeness." ([1])

  • Hilarious SMBC comic on privacy and technology ([1])

  • Great SMBC comic: "Wanna play the Bayesian drinking game?" ([1])

  • Hilarious John Oliver segment on net neutrality ([1]) directs people to FCC website to comment, crashing FCC website ([2])

  • Very funny, from The Onion: "New Facebook Feature Scans Profile To Pinpoint Exactly When Things Went Wrong" ([1])

Wednesday, June 04, 2014

Project Euler and blending math and computer science for education

Project Euler is a simple and surprisingly good educational tool for a blend of computer science and math. Highly recommended.

You are given a problem (good examples: [1] [2] [3] [4]), go off and work on it in whatever programming language you like using whatever tools you like, and submit your answer (multiple submissions allowed). Simple, but surprisingly fun and interesting.

It's been around for a while (since 2006), and, though I've looked at it a few times, I only recently got addicted to it. It's not, as I first thought, just a series of interview-style coding questions, but a much more interesting set of deeper challenges in math that require programming to explore and solve. It's a great way to refresh on math and fun too.

Honestly, I can't say enough good things about it. I've blown hundreds of hours on some addictive video game before, addicted to the point that it occasionally interferes with work and sleep even, and this has the same feel. It's a great little educational tool and fun as well.

Definitely worth a look. Seems like it'd work for older teenagers too if you're looking for a summer project for a teen that already has some programming skill.

Friday, May 02, 2014

More quick links

More of what has caught my attention lately:
  • Excellent history of Google's love-hate relationship with management (hint: it's mostly hate) ([1])

  • Excellent BBC documentary on Amazon.com, plenty of fun historical tidbits, quite critical in parts, very well done ([1])

  • Excellent charts on the history of wealth concisely summarizing three centuries ([1])

  • Compelling example of virtual tourism ([1])

  • Visually stunning math concepts which are easy to explain ([1])

  • "The most interesting things are happening at the intersection of two fields" ([1])

  • "The largest driver of Facebook’s mobile revenue is app-install ads ... largely purchased by free-to-play game publishers such as King (maker of Candy Crush Saga) and Big Fish Games (the Bejeweled series) ... to target the small percentage of players who will spend hundreds of dollars on in-app purchases." ([1] [2])

  • "When you subtract out the value of Yahoo's stake in Alibaba, the rest of Yahoo is worthless. Indeed, it has negative worth." ([1] [2] [3])

  • "Newspaper print ad revenue has declined 73% in 15 years" ([1])

  • "Microsoft is backtracking on practically every part of the Windows 8 interface that developers abhorred" ([1] [2] [3] [4])

  • Survivor bias in perceptions of startup life and what might be closer to reality: "It's a decision to throw away a large chunk of your precious youth at a venture which is almost certain to fail" ([1] [2])

  • "Many hospitals in the US still use Windows XP on workstations and healthcare devices" ([1])

  • "It's no longer realistic to think that routers, DVRs, or other Internet-connected home appliances aren't worth an attacker's time ... poorly designed 'Internet of Things' devices ... [are] particularly easy to hack" ([1])

  • Newegg exec on patent trolls: "Why those asshats continue to trade at ANY value, I do not know. The world would be a better place without them." ([1])

  • I still don't understand why more tech companies don't provide free food ([1] [2])

  • Humor: "The pain of being the only engineer in a business meeting" ([1])

Wednesday, April 02, 2014

Quick links

What has caught my attention lately:
  • Dilbert on A/B testing: "Bend to my will and choose the orange button, you mindless click-puppets!" ([1])

  • Major performance increases on smartphones are disappearing, which will slow sales and reduce revenues ([1] [2])

  • Price war in cloud services ([1])

  • On Facebook buying Oculus: "The dominant reaction to the move could be summed up in three letters: WTF" ([1] [2])

  • Remember this? "Companies could cause their stock prices to increase by simply adding an 'e-' prefix to their name or a '.com' to the end, which one author called 'prefix investing'" ([1] [2])

  • VCs favor pitches from attractive men ([1] [2])

  • "We've known for a while that email providers could look into your inbox, but the assumption was that they wouldn't" ([1] [2])

  • Bad new trend: Apps that covertly mine Bitcoins for someone else ([1] [2])

  • More companies should do this: Run large scale surveys of employees to discover what makes people happy and productive ([1])

  • Combining dissimilar fields is hard, but can also lead to discovering lots of low hanging fruit (at least from where you are standing) that no one else has picked ([1])

  • Good idea from a recent Google paper: Mine the web to build up knowledge of objects that are likely and unlikely to co-occur, then use that to accept or reject candidates during object recognition ([1] [2])

  • Cool throwback idea from a recent MSR paper: Old school circuit-switched networks in the data center using cheap commodity FPGAs ([1] [2])

  • “There doesn't need to be a protective shell around our researchers where they think great thoughts" ([1] [2])

  • Surprisingly compelling results: Generate likely 3D models of facial appearance solely from DNA ([1] [2])

  • Stem cells used to grow strong muscles that repair themselves when damaged ([1])

  • The ancient Greeks and Persians had to occasionally fight off lions ([1] [2])

  • Great visualization of conditional probability ([1])

  • Galleries of hilariously useless items ([1] [2])