Lincoln's blog

Value Systems, AI and Human Existence

The Singularity Institute has recognized that nonhuman intelligences can have -- or evolve -- arbitrary value systems. And arbitrary value systems seem likely to be incompatible with ours: human value systems are generally based on continuing human existence, which seems fairly constraining in the scope of all value systems. For example, any value system which wants to consume lots of a resource that humans depend on (air, sunlight, carbon atoms, habitable planets, ...) will probably conflict with most human values.

So why do we still exist? No intelligence has yet succeeded in consuming our resources. We've not even managed to find convincing evidence of the existence any other intelligence which can compete for the resources we need. Lucky. (Or apply the anthropic principle.)

However, people are working on improving artificial intelligence in software. Currently AI is not very smart or powerful. But contrasted with our own brains, AI is very flexible -- it doesn't come with any assumptions or value systems and it is easy to change by editing the source code. It's easy to imagine an AI which could edit its own source code. ¹

Given that an AI is editing its own code, and doing so in an intelligent manner, with the goal of making itself more intelligent, there is no obvious limit on the rate of improvement and scope of intelligence that such a self-improving entity could achieve. This is what's known as the technological singularity -- an AI that improves itself rapidly until it vastly outstrips human intelligence.

Up to this point, the only thing we are assuming about an intelligent AI's value system is that it wants to make itself more intelligent. Without any other assumptions, this isn't likely to be compatible with the human value of continuing existence -- for example, to become more intelligent, perhaps the AI needs all the sunlight hitting earth for energy to run more brain cells. Since the AI is so much smarter than us, it would outsmart us to get itself to that point, extinguishing the human race in the process. Oops.

Having followed this train of thought, and considering it a serious risk to continuing human existence, the Singularity Institute was formed. Its purpose is to mitigate the risk of human extinction by technological singularity. Rather than trying to prevent a singularity from occurring, which seems unlikely to work², SIAI is trying to figure out how to make an AI whose values are sufficiently aligned with human values such that singularity doesn't pose significant risk of human extinction.

I recently learned that SIAI is sponsoring Visiting Fellows -- essentially an internship at the Institute to work on these problems. As part of my application, they asked me to explain SIAI's purpose and how I can contribute. Having done the former, I will attempt to answer the latter in the next blog post (linked here when it's ready).

posted at: 15:44 | path: /self | permanent link to this entry

25 Things

There's this Facebook virus going around: "Once you've been tagged, you are supposed to write a note with 25 random things, facts, habits, or goals about you. At the end, choose 25 people to be tagged."

Effective? Yes. Everybody loves an opportunity to write a long post about themselves! I had fun reading a couple instances of the meme by some friends, so here ya go. I'm not going to "tag" anyone except for the people who tag me, because I still ethically refuse to spread the virus. If you wish to be tagged, let me know (though if you're reading this, I don't know why you need that...)

1. The number one thing that makes me happy is interacting with friends. Laughter, helping someone out, cooperating to do something amazing, and falling in love have undoubtedly been the most satisfying experiences in my life, and I never expect this to change. People are the entire meaning of life to me.
2. Secondarily, what makes me happy is learning (through games, computer programming, reading, etc.) I have high expectations for myself being able to learn things quickly and well, and I wish to keep it up. When I stop learning in a game, I stop enjoying it. Same for programming projects. My enjoyment in classes tended to be proportional to the rate I was learning from them (of course, there were variations, but I think this is a generally true trend). It explains why I hated long homework assignments: I never felt like the learning to work ratio was high enough for it to be fun. But I usually enjoyed the genuinely difficult homeworks.
3. I have dreams of making a difference to lots of people. I don't know whether I can do this directly through starting a company, but it is always one of my goals when I start one. If not, I want to become independently wealthy and use my money to allow me to work on projects which can make a difference to people. I guess I will admit I also want to be remembered, although if I made a difference and nobody knew it but me, I would be satisfied.
4. I ski. Fast. Straight down.
5. I strive to have more self control. My friends criticize me for not knowing what the hell is going on with myself and I think they're right. I have bad habits that I can't quit. I am easily distracted from work (even important, satisfying work) by short-term desires which won't make me a better person. My goals in this are to give myself a little negative feedback when I notice I'm out of control, by writing them down.
6. Tom Mendoza, the president of NetApp, came to speak to the interns a few times when I worked there. He always gave the same speech: improve yourself and you can achieve great things. Small steps are the key to success; you need to set goals for yourself, but not abstract ones -- they must be concrete. "Time- bound and measurable" was the phrase he drummed into us. And I would say he is a great man. So I'm setting myself time-bound and measurable goals.
7. OK, onto less weighty matters. I have an interesting relationship with music. I took violin lessons at my mother's insistence from when I was two until when I was eighteen (!!). (Technically I switched to viola for the last two or three years of lessons, but that was barely different.) I never really liked it, but it was always part of my life. Went off to college and, because of momentum, joined a chamber group first semester. It was pretty good; I always like playing in groups. Of course, I stopped after that because my momentum ran out and I never really felt like I wanted to practice. (Unlike programming or gaming, I don't feel like I learn very quickly, so it's not as fun.) I recently got ahold of a very cheap electric violin and I've been practicing again! I'm much more satisfied playing it than classical violin, and I'm also less self-conscious (especially in an apartment where my roommates and neighbors hear me much less with a quiet instrument).
8. Maybe it's that I feel better about playing music I like to listen to on violin. I listen to a good amount of electronic music -- good for programming -- as well as some rock, with the occasional pop thrown in. Video game music is often electronic or classical sounding, and I know a lot of it by heart (and it can be fun to play too!) Yasunori Mitsuda is my favorite video game music composer.
9. I notice that music can affect my mood a lot. When a movie has good music, it tremendously amplifies the experience; Slumdog Millionare was a recent movie which had really excellent music. It's so rare to find good movies with good music. Another recent one was Kill Bill. I also put on music when I'm doing a depressing task, like cleaning the kitchen by myself, and it can often pick me up and make me excited to do it.
10. I have glasses and I am quite happy about that. I felt like they were part of my identity before I actually had them, so it was basically a relief in 8th grade when I got them. For some reason, glasses connote a certain personality type that I want to be (and incidentally one I tend to be attracted to as well).
11. Memorizing lyrics to songs is really entertaining, especially if you actually perform the song for people. I can do "Star Wars Gangsta Rap" and "It's All About the Pentiums" (both are rap, which is even more hilarious) by heart. In high school, my friends and I used to do the Gangsta Rap for nerds in Magic card shops, who always looked at us funny. But we didn't care what anybody else thought, even other nerds.
12. Richard Feynman is my hero. Based on his autobiography ("Surely You're Joking, Mr. Feynman"), he thinks about the world similarly to how I do. His thought processes seem to be based on strong, intuitive mental models, which is exactly how I understand everything. Also, one of my favorite lines from one of his stories was "What do YOU care what other people think?" It became my personal motto.
13. I accidentally the whole number thirteen. Is that bad?
14. I don't use Gmail. Almost all my friends use it as their main email account. I'm successfully resisting it though!
15. I bake bread. You can read about it elsewhere on my blog. It's an instance of bounded creativity -- you can go crazy and try changing the recipe all around, but it won't work if constraints aren't met and it's quite delicate, so there's always some risk and learning. It also has some aesthetic aspects -- getting it cooked an attractive color, getting a nice grigne and so on. And it's one of my favorite things to eat. Always nice when you can cook and make yourself the happiest.
16. Things I hate: driving cars, unfair assumptions. For different reasons. And there are probably other things too, but that's as far as I got before I decided to go onto the next question.
17. When I was younger, I hated rice, pasta, and anything spicy. My brother hated tomatoes, mushrooms, and milk. At the time I thought he was the crazy one but I am now pretty sure I was crazier. Now I pretty much just hate cilantro. (It's genetic, so I have an excuse.) There are other foods I don't like very much because they're boring, but I've been hungry enough to gain an appreciation for many foods I didn't used to like. (Is that legitimate grammar?)
18. I have an active profile on a dating site. Mostly it sucks. I've talked to several girls who were attractive, and met a couple of them in real life, but one or both of us always lost interest. It can be quite addictive to browse people's profiles on the site (maybe the NEXT one will be perfect!) But it's really hard for me to be attracted to someone I haven't met, so the most initially interesting profiles are ones who share my interests, and profiles that don't give a good first impression (all capital letters anyone?) are right out -- it's a bad first impression and that's really all you get. I feel a bit bad profiling like that, but it's to be expected I think. Meh.
19. I've never smoked pot or used any other drug illegally (except for alcohol before I was 21). I don't intend to. This is an interesting part of my value system because I don't really understand it. I do think marijuana should be legalized. I might even try it then; we'll see.
20. I've had good luck with backpacks. I've had the same backpack for five years (all through college). The one before that, I had only for a year or so, but the one before THAT I had for another four years or something. Different brands, too; I don't know why they are so good for me. I don't treat them especially well (or poorly, for that matter), but I use them a lot.
21. Computer Science was easy for me. I had the "computer science is hard" complex while I was in school, because everyone else in the department had that sense of elitism, but when I look back at it, it was a pretty easy major. I'm not saying this to brag but just to think "maybe I didn't get the most out of my college experience?" It's a bit unfortunate. I'm still thinking of going to grad school. Hopefully it'll be hard then.
22. I have a beard, for the first time in my life. It's not a part of my identity, although sometimes when I look in the mirror I see a different person with that beard and think "Oh, I could be him". Weird.
23. I admire persistence. I don't think of it as a trait I normally have, although I can sometimes dredge up a good reserve of persistence for weird shit (like doing stuff I'm not supposed to be able to do -- there's a certain room in Metroid Prime that you need like two or three items to climb and I didn't have either one and I figured out how to climb it, but it took me a couple hours and a LOT of tries.)
24. I'm usually optimistic. Sometimes it gets on people's nerves. But it makes me happier overall, because I have less sadness being disappointed than being worried.
25. I think of the daily shower as one of the greater pleasures in the modern lifestyle. It's warm, comfortable, nobody can bother you and I occasionally even solve an interesting problem. I always look forward to showers.

posted at: 05:17 | path: /self | permanent link to this entry

The role of my blog

I figured out that, in large part, the role of my blog is to provide myself with a place to describe certain technical things that I am interested in, and describe them in a way that other people can understand them. I do this because, as I describe something, it forces me to learn it well enough to describe it, and thus my blog is really helping me. You are just a tool in my scheme to understand everything.

Thanks to the people who read this (not many, but there's a few a week.)

posted at: 16:40 | path: /self | permanent link to this entry

Bermuda

I am not going to talk about all aspects of my Bermuda trip, but I would like to highlight two pieces of it. To talk about everything would take hours...

Mopeds are awesome. Tons of people there have them, and they are cool, smooth, strong, fast, powerful and sweet! When you visit, the Thing To Do is rent a moped and drive it all over the island looking for stuff. You can get from one end to the other in an hour on a moped, and it's really enjoyable to drive the things -- the only thing to remember is that you gotta stay on the left side of the road! Anyway, it's tons of fun, and I recommend it.

One other thing which I would like to talk about is these people who live here, whose house we visited last night -- the Bordens. They are extremely rich and built their own huge house on a cliff in Bermuda -- looking south over the Atlantic from Tucker's Town (on the south coast of Bermuda). This house totally blew me away. It was constructed from shellstone imported from Texas -- it's this stone that is filled with imprints from shells, and it occurs naturally in Texas, I guess. Anyway, this was the place where you would expect to see stylish people dressed in golf shirts and berets, lounging in colorful armchairs, contemplating the problems of the world, working wirelessly with their Powerbooks. Sure enough, there were colorful armchairs (in a huge room with abstract art, brightly colored carpets and densely windowed walls on both sides). There were Powerbooks, there were golf shirts, there was wireless. It was truly a beautiful thing to behold.

The awesome thing was that the people were really intelligent and insightful, and we talked about really high-brow topics: creationism vs. evolution, neuroscience and machine learning, the past and future of the world. I would have called it pretentious, except there was no pretense -- these people really cared about these things -- this is what they did all day. Their "driveway" is golf-green-quality grass -- it's maintained about 5mm thick, with fiberglass fibers in the soil to help keep the roots in.

Anyway, I had a happy birthday on Friday, thank you, and I'm 19 now. Growing up is kinda crazy. But lots of fun.

posted at: 12:45 | path: /self | permanent link to this entry