Now that she was home, she meant to plant them for remembrance. 

While there are many themes, ideas, and messages within Janie as a character and Their Eyes Were Watching God as a book but perhaps the most prevalent throughout is the focus on Janie's journey for independence and how it shapes her from the start to the beginning. Over time her character develops massively and it is often times very hard to encapsulate a development like that in a single drawing without multiple separated parts. I wanted my project to embody this idea of a journey for independence and change in both of those (the very start and the very beginning) points in the novel, with only one image, the tree.

Most times I pretend I ain't there

There are hundreds of different images of the child.Each one of you has inside yourself an image of thechild that directs you as you begin to relate to a child.This theory within you pushes you to behave incertain ways; it orients you as you talk to the child,listen to the child, observe the child. It is verydifficult for you to act contrary to this internal image.For example, if your image is that boys and girls arevery different from one another, you will behavedifferently in your interactions with each of them.

But in a few Weeks After resolv'd to run.”

al­though enca ed in what seem a olid, impenetrable armor,the lobster can receive stimuli and impression from withoutas readily as ifit possessed a soft and delicate skin

d, as to the faculties of the mind, setting aside the arts grounded upon words and especially that skill of proceeding upon general and infallible rules called science, which very few have and but in few things, as being not a native faculty born with us, nor attained, as prudence, while we look after somewhat else, I find yet a greater equality amongst men than that of strength. For prudence is but experience, which equal time equally bestows on all men in those things they equally apply themselves unto. That which may perhaps make such equality incredible is but a vain conceit of one’s own wisdom, which almost all men think they have in a greater degree than the vulgar, that is, than all men but themselves, and a few others whom by fame or for concurring with themselves they approve.

Patton worries that the technology being proffered to schools may be more likely to misfire on language used by black youth, potentially causing them to experience greater scrutiny from school administrators.

Resources on Design Thinking in Different Industries不同行业的设计思维资源 Since we know you come from many industries, we wanted to provide some industry-specific optional readings for you. Here are some resources for design thinking in a variety of industries you may want to explore.因为我们知道你来自许多行业，我们想为你提供一些行业特定的可选读物。 这里有一些你可能想要探索的各种行业的设计思维的资源。 Education教育 "Design Thinking for Educators 教育工作者的设计思维" from IDEO 来自 IDEO "Design Thinking in Education 教育中的设计思维" from Harvard University 来自哈佛大学 "Design Thinking in Pedagogy 教育学中的设计思维" by Ineta Luka 作者: Ineta Luka "Design and Design Thinking in Business and Management Higher Education 工商管理类高等教育中的设计与设计思维" by Judy Matthews and Cara Wrigley 作者: Judy Matthews and Cara Wrigley Healthcare医疗 The Institute of Healthcare Design Thinking 医疗保健设计思维研究所 website 网站 "Design Thinking in Health Care: Ear Wax, Hemolysis, and Other Opportunities 保健中的设计思维: 耳垢、溶血和其他机会'' by Benjamin Doolittle, MD 本杰明杜利特尔医学博士 "Design Can Improve Healthcare; Can it Also Lead to New Cures? 设计可以改善医疗保健，也可以带来新疗法吗？" by David A. Shaywitz 作者: David a. Shaywitz "Design Thinking for Doctors and Nurses 医护人员的设计思维" by Amitha Kalaichandran 作者: Amitha Kalaichandran Government政府 "Design Thinking: The Answer to the Impasse Between Innovation and Regulation 设计思维: 创新与规制僵局的出路" by Alice Armitage, Andrew K. Cordova, and Rebecca Siegel 作者: Alice Armitage，Andrew k. Cordova，Rebecca Siegel "Design Thinking for Government Services: What Happens When the Past Limits our Vision of the Future? 政府服务的设计思维: 当过去限制我们对未来的展望时会发生什么？" by Arturo Muente-Kunigami 作者: Arturo Muente-Kunigami "Managers as Designers in Local Government 地方政府中的设计师管理者" by Lancing Farrell 作者: Lancing Farrell Libraries and Museums图书馆及博物馆 "Design Thinking: Librarians Are Incorporating it Into Their Practice “设计思维: 图书馆员将其融入实践" by Steven Bell 作者: Steven Bell "Critiques Help Us Think Critically about Design Thinking 批判帮助我们对设计思维进行批判性思考" by Steven Bell 作者: Steven Bell Design Thinking for Libraries 图书馆的设计思维 website 网站 Design Thinking for Museums 博物馆的设计思维 website 网站 "Human Centered Design 以人为本的设计" from Derby Museums 来自德比博物馆 Human Resources人力资源 "HR and the Modern Workplace: Using Design Thinking to Create Successful Digital Transformation 人力资源与现代工作场所: 用设计思维创造成功的数字化转型" by Caterina Sanders 作者: Caterina Sanders "How Design Thinking is Helping HR Create Engaging Employee Experiences 设计思维如何帮助人力资源创造有吸引力的员工体验" by Linda Naiman 作者: Linda Naiman "Design Thinking: Crafting the Employee Experience 设计思维: 打造员工体验" by Josh Bersin, Marc Solow, and Nicky Wakefield 作者: Josh Bersin，Marc Solow，Nicky Wakefield Management管理 "Why Managers Should Consider Adopting a Design Thinking Approach 为什么管理者应该考虑采用设计思维方法" by Nicole Ferrer 作者: Nicole Ferrer "Managers as Designers in Local Government 地方政府中的设计师管理者" by Lancing Farrell 作者: Lancing Farrell "Creating Breakthrough Innovations Through Design with Big and Small Data Analysis 利用大小数据分析进行设计创新" By Soren Petersen 作者: 索伦 · 彼得森 "An Introduction to Design Thinking for Innovation Managers 创新管理者的设计思维导论" by Paul Hobcraft 作者: Paul Hobcraft Service Design服务设计 "Service Design Thinking - The Next Frontier of a Great Workplace Experience 服务设计思维——伟大工作体验的下一个前沿" from the International Service System (ISS) Group ” ，由国际服务体系小组提供 "The Principles of Service Design Thinking - Building Better Services 服务设计思维原则——营造更好的服务" from the Interaction Design Foundation 来自交互设计基金会 "Service Design 101 服务设计101" by Sarah Gibbons 作者: Sarah Gibbons "6 Principles of Service Design to Help You Reach Your Customers 6个服务设计原则帮助你接触你的客户" by Maria Hayhow

Before the end, one began to pray to it; inherited instinct taught the natural expression of man before silent and infinite force

information is much more widely avail-able from people who have strong political, economic, religious, or ideological stances that profoundly influence the nature of the information they present to others. As a result, we must assist students to become more critical consumers of the information they encounter

"(It) could take six months, a year, two years, I don't know, but we can't really wait that long for Chantelle," he said, fighting back tears. "We've got to get it sooner."

We make and use a lot of stuff—and stuffmatters.

Understand is that critical piece – it’s the set of skills that help us comprehend, contextualize, and critically evaluate digital media so that we can make informed decisions about what we do and encounter online. These are the essential skills that we need to start teaching our kids as soon as they go online.

This is certainly something we will need to grapple with collectively as we struggle to find the right balance between ensuring the protection and well-being of individuals while maintaining liberty and independence.

the phenomenal form

Every couple of years, mainstream media hacks pretend to have just discovered libertarianism as some sort of radical, new and dynamic force in American politics. It’s a rehash that goes back decades, and hacks love it because it’s easy to write, and because it’s such a non-threatening “radical” politics (unlike radical left politics, which threatens the rich). The latest version involves a summer-long pundit debate in the pages of the New York Times, Reason magazine and elsewhere over so-called “libertarian populism.” It doesn’t really matter whose arguments prevail, so long as no one questions where libertarianism came from or why we’re defining libertarianism as anything but a big business public relations campaign, the winner in this debate is Libertarianism. Pull up libertarianism’s floorboards, look beneath the surface into the big business PR campaign’s early years, and there you’ll start to get a sense of its purpose, its funders, and the PR hucksters who brought the peculiar political strain of American libertarianism into being — beginning with the libertarian movement’s founding father, Milton Friedman. Back in 1950, the House of Representatives held hearings on illegal lobbying activities and exposed both Friedman and the earliest libertarian think-tank outfit as a front for business lobbyists. Those hearings have been largely forgotten, in part because we’re too busy arguing over the finer points of “libertarian populism.” Milton Friedman. In his early days, before millions were spent on burnishing his reputation, Friedman worked as a business lobby shill, a propagandist who would say whatever he was paid to say. That's the story we need to revisit to get to the bottom of the modern American libertarian "movement," to see what it's really all about. We need to take a trip back to the post-war years, and to the largely forgotten Buchanan Committee hearings on illegal lobbying activities, led by a pro-labor Democrat from Pennsylvania, Frank Buchanan. What the Buchanan Committee discovered was that in 1946, Milton Friedman and his U Chicago cohort George Stigler arranged an under-the-table deal with a Washington lobbying executive to pump out covert propaganda for the national real estate lobby in exchange for a hefty payout, the terms of which were never meant to be released to the public. They also discovered that a lobbying outfit which is today credited by libertarians as the movement’s first think-tank — the Foundation for Economic Education — was itself a big business PR project backed by the largest corporations and lobbying fronts in the country. It starts just after the end of World War Two, when America’s industrial and financial giants, fattened up from war profits, established a new lobbying front group called the Foundation for Economic Education (FEE) that focused on promoting a new pro-business ideology—which it called “libertarianism”— to supplement other business lobbying groups which focused on specific policies and legislation. The FEE is generally regarded as “the first libertarian think-tank” as Reason’s Brian Doherty calls it in his book “Radicals For Capitalism: A Freewheeling History of the Modern Libertarian Movement” (2007). As the Buchanan Committee discovered, the Foundation for Economic Education was the best-funded conservative lobbying outfit ever known up to that time, sponsored by a Who’s Who of US industry in 1946. A partial list of FEE’s original donors in its first four years— a list discovered by the Buchanan Committee — includes: The Big Three auto makers GM, Chrysler and Ford; top oil majors including Gulf Oil, Standard Oil, and Sun Oil; major steel producers US Steel, National Steel, Republic Steel; major retailers including Montgomery Ward, Marshall Field and Sears; chemicals majors Monsanto and DuPont; and other Fortune 500 corporations including General Electric, Merrill Lynch, Eli Lilly, BF Goodrich, ConEd, and more. The FEE was set up by a longtime US Chamber of Commerce executive named Leonard Read, together with Donaldson Brown, a director in the National Association of Manufacturers lobby group and board member at DuPont and General Motors. That is how libertarianism in America started: As an arm of big business lobbying. Before bringing back Milton Friedman into the picture, this needs to be repeated again: “Libertarianism” was a project of the corporate lobby world, launched as a big business “ideology” in 1946 by The US Chamber of Commerce and the National Association of Manufacturers. The FEE’s board included the future founder of the John Birch Society, Robert Welch; the most powerful figure in the Mormon church at that time, J Reuben Clark, a frothing racist and anti-Semite after whom BYU named its law school; and United Fruit president Herb Cornuelle. The purpose of the FEE — and libertarianism, as it was originally created — was to supplement big business lobbying with a pseudo-intellectual, pseudo-economics rationale to back up its policy and legislative attacks on labor and government regulations. This background is important in the Milton Friedman story because Friedman is a founding father of libertarianism, and because the corrupt lobbying deal he was busted playing a part in was arranged through the Foundation for Economic Education. According to Congressional hearings on illegal lobbying activities 1946 was the year that Milton Friedman and his U Chicago cohort George Stigler arranged an under-the-table deal with a Washington lobbying executive to pump out covert propaganda for the national real estate lobby in exchange for a hefty payout, the terms of which were never meant to be released to the public. The arrangement between Friedman and Stigler with the Washington real estate lobbyist was finally revealed during a congressional review of illegal lobbying activities in 1950, called the Buchanan Committee. Yes, there was something called accountability back then. I only came across the revelations about Friedman’s sordid beginnings in the footnotes of an old book on the history of lobbying by former Newsweek book editor Karl Schriftgiesser, published in 1951, shortly after the Buchanan Committee hearings ended. The actual details of Milton Friedman’s PR deal are sordid and familiar, with tentacles reaching into our ideologically rotted-out era. False, whitewashed history is as much a part of the Milton Friedman mythology as it is the libertarian movement’s own airbrushed history about its origins; the 1950 Buchanan Committee hearings expose both as creations of big business lobby groups whose purpose is to deceive and defraud the public and legislators in order to advance the cause of corporate America. The story starts like this: In 1946, Herbert Nelson was the chief lobbyist and executive vice president for the National Association of Real Estate Boards, and one of the highest paid lobbyists in the nation. Mr. Nelson’s real estate constituency was unhappy with rent control laws that Truman kept in effect after the war ended. Nelson and his real estate lobby led what House investigators discovered was the most formidable and best-funded opposition to President Truman in the post-war years, amassing some $5,000,000 for their lobby efforts—that’s $5mln in 1946 dollars, or roughly $60 million in 2012 dollars. So Herbert Nelson contracted out the PR services of the Foundation for Economic Education to concoct “third party” propaganda designed to shore up the National Real Estate lobby’s legislative drive — and the propagandists who took on the job were Milton Friedman and his U Chicago cohort, George Stigler. To understand the sort of person Herbert Nelson was, here is a letter he wrote in 1949 that Congressional investigators discovered and recorded: "I do not believe in democracy. I think it stinks. I don’t think anybody except direct taxpayers should be allowed to vote. I don’t believe women should be allowed to vote at all. Ever since they started, our public affairs have been in a worse mess than ever." It’s an old libertarian mantra, libertarianism versus democracy, libertarianism versus women’s suffrage; a position most recently repeated by billionaire libertarian Peter Thiel — who was Ron Paul’s main campaign funder in his 2012 presidential campaign. So in 1946, this same Herbert Nelson turned to the Foundation for Economic Education to manufacture some propaganda to help the National Association of Real Estate Boards fight rent control laws. Nelson chose to work with the FEE because he knew that the founder of the first libertarian think-tank, Leonard Read, agreed with him on a lot of important issues. Such as their mutual contempt for democracy, and their disdain for the American public. Leonard Read, the legendary (among libertarians) founder/head of the FEE, argued that the public should not be allowed to know which corporations donated to his libertarian front-group because, he argued, the public could not be trusted to make “sound judgments” with disclosed information: "The public reporting would present a single fact—the amount of a contributor’s donation—to casual readers, persons having only a cursory interest in the matter at issue, persons who would not and perhaps could not possess all the facts. These folks of the so-called public thus receive only oversimplifications or half-truths from which only erroneous conclusions are almost certain to be drawn. If there is a public interest in the rightness or wrongness of corporate or personal donations to charitable, religious or education institutions, and I am not at all ready to concede that there is, then that interest should be guarded by some such agency as the Bureau of Internal Revenue, an agency that is in a position to obtain all the facts, not by Mr. John Public who lacks relevant information for the forming of sound judgments…Public reporting of a half-truth is indeed a significant provocation." So in May 1946, Herbert Nelson of the Real Estate lobby, looking for backup in his drive to abolish federal rent control laws on behalf of landlords, contacted libertarian founder Leonard Read of the FEE with an order for a PR pamphlet “with some such title as ‘The Case against Federal Real Estate Control’,” according to Karl Schriftgiesser’s book The Lobbyists. What happened next, I’ll quote from Schriftgiesser: "They were now busily co-operating on the new project which the foundation had engaged Milton Friedman and George J. Stigler to write. It was to be called Roofs and Ceilings and it was to be an outright attack on rent controls. When Nelson received a copy of the manuscript he wrote Read to say, “The pamphlet…is a dandy. It is just what I wanted." The National Association of Real Estate Boards was so pleased with Milton Friedman’s made-to-order propaganda that they ordered up 500,000 pamphlets from the FEE, and distributed them throughout the real estate lobby’s vast local network of real estate brokers and agents. In libertarianism’s own airbrushed history about itself, the Foundation for Economic Education was a brave, quixotic bastion of libertarian “true believers” doomed to defeat at the all-powerful hands of the liberal Keynsian Leviathan and the collectivist mob. Here is how libertarian historian Brian Doherty describes the FEE and its chief lobbyist Leonard Read: "[Read] would never explicitly scrape for funds… He never directly asked anyone to give anything, he proudly insisted, and while FEE would sell literature to all comers, it was also free to anyone who asked. His attitude toward money was Zen, sometimes hilariously so. When asked how FEE was doing financially, his favorite reply was, “Just perfectly.”… Read wanted no endowments and frowned on any donation meant to be held in reserve for some future need." And here is what the committee’s own findings reported—findings lost in history: "It is difficult to avoid the conclusion that the Foundation for Economic Education exerts, or at least expects to exert, a considerable influence on national legislative policy….It is equally difficult to imagine that the nation’s largest corporations would subsidize the entire venture if they did not anticipate that it would pay solid, long-range legislative dividends." Or in the words of Rep. Carl Albert (D-OK): "Every bit of this literature is along propaganda lines." The manufactured history about libertarian’s origins, or its purpose, parallels the manufactured myths about one of big business’s key propaganda tools, Milton Friedman. As the author of The Lobbyists, not knowing who Milton Friedman was at the time, wrote of Friedman’s collaborative effort with Stigler: “Certainly [the FEE’s] booklet, Roofs or Ceilings, was definitely propaganda and sought to influence legislation….This booklet was printed in bulk by the foundation and half a million copies were sold at cost to the National Association of Real Estate Boards, which had them widely distributed throughout the country by its far-flung network of local member boards.” There's no idealism here. The notion that libertarian ideas have captured the political imagination of millions in this country is a root problem: if we're going to escape the corporate oligarchy that is running this country–their ideas can't possibility be the alternative solution. This movement has to be recognized for what it is.

Essential questions about our humanity, even questions of meaning, once mostly the province of the humanities, are now part of scientific research.

We define ourselves through our cell phones and think of scenarios where we will be different, part human part machine, with some scientists even speculating that a singularity point will be reached when machines will become smarter than we are.

Within the "Frequently Asked Questions" section on gojo.com, the agency noted the company says that "Purell Healthcare Advanced Hand Sanitizers, which are formulated with ethyl alcohol, may be effective against viruses such as the Ebola virus, norovirus, and influenza."

He demonstrated that persons may have power and high position and yet be deeply confused and irrational. He established the importance of asking deep questions that probe profoundly into thinking before we accept ideas as worthy of belief.

annoyances; irritationsmysteries; secreciesproblems; sicknessessymptoms; signsWhich sentence states a central idea of paragraphs 17-20?California almond farmers rotate their crops between almonds and grasslands in order to utilize rented bee colonies to pollinate their plants and almonds.Scientists at the USGS studied summertime land use in North Dakota, South Dakota, and Minnesota to determine whether or not rented apiaries were beneficial to California almonds.Scientists have found that land converted from grasslands to row crops resulted in smaller and less profitable bee colonies, which in turn affected the almond crop in California.The profitability of the beekeeping industry is a direct result of careful observation by local scientists, the results of which affect state and federal land-use laws and practices.How does the author use structure to support the idea that bees require specific diets?by providing the cause and effect process of how bees are nourished by apiariesby using description and analogy to show how a limited human diet has health consequencesby comparing and contrasting the specific nutritional makeup of different types of available plants and pollen by explaining the problems associated with certain types of plants and solutions for how to plant more nutritious grasses for beesWhich claim lacks evidence in the passage to support it?“…critical honey bee and wild bee populations in the United States have been declining in recent years, creating concern about the future security of pollination services for agricultural crops.” (paragraph 3)“Originally imported from Europe in the 17th century, honey bees are essential to maintaining food production in North America today.” (paragraph 5)“Honey bees in the Northern Great Plains are of critical importance to the commercial beekeeping industry and to the diversity of U.S. crops.” (paragraph 10)“While the importance of a healthy pollinator population to agriculture is clear, pollinators are just as important to sustaining functioning ecosystems and food supplies for wildlife.” (paragraph 21)What is the meaning of the word jeeringly as it is used in paragraph 3? with ignorance; unknowinglywith embarrassment; quietlywith ease; simplywith disdain; insultinglyPart A What is a theme of the passage? Participating in team sports is an important part of childhood.One should not judge another by his or her outer appearance.New students should be treated with kindness and respect.Sometimes friends disagree about important matters.Part B Which quotation from the passage supports the answer in Part A? “‘Plenty of fellows, apparently,’ answered the third member of the group, Wall Clausen, hurrying to prevent the threatening quarrel.” (paragraph 11)“Some sixty boys, varying in age from fifteen to nineteen, some clothed in full football pads, some wearing the ordinary dress…all laughing or talking with high spirits were standing around the field.” (paragraph 19)“He stepped back, balancing the ball in his right hand, took a long stride forward, swung his right leg in a wide arc, dropped the ball, and sent it sailing down the field toward the distant goal. A murmur of applause took the place of sarcastic laughter…” (paragraph 33)“‘Yes, that’s pretty fair. Some day with hard practice you may make a kicker. What class are you in?’” (paragraph 34)Read the sentences from paragraphs 4 and 30.The subject of their mocking remarks, although standing a short distance away, apparently heard none of them. (paragraph 4) Several audible remarks were made, none of them at all flattering to the subject of them; but if the boy heard them he made no sign but accepted the ball from Blair without fumbling it, much to the surprise of the onlookers. (paragraph 30)Based on the sentences, which words describe Joel March?dismayed; hurtfocused; composedthoughtful; reflectiveunfriendly; detachedHow does paragraph 19 contribute to the plot?It suggests that some of the boys were more suited to trying out than others.It explains the importance of preparedness when playing an activity such as football.It develops the contrast between the crowd of excited boys and the isolated and quiet new student. It illustrates the idea that Bart Cloud wants to exclude Otto West from tryouts just as he wants to exclude the new student.How does Joel March respond to football tryouts?He is nervous and afraid to join the group.He is calm and wants to show off his skills.He is arrogant and embarrasses the others.He is comfortable and makes friends easily.Read paragraph 23. Blair, suppressing a smile of amusement as he looked the applicant over, asked: “Ever played football?” How does the paragraph convey Wesley Blair’s feelings?  Wesley Blair is excited to have such a tall player try out for football.Wesley Blair feels pleased and honored to have a player ask for help.Wesley Blair is surprised and entertained by Joel March’s appearance.Wesley Blair feels Joel March is tricking him by pretending to know how to play football.How does the contrast between lines 49-52 and lines 69-72 contribute to the poem’s theme?The contrast suggests that being happy is a matter of priorities.The contrast suggests that one must find happiness in others.The contrast suggests that there are many paths to happiness.The contrast suggests that happiness is found in tasks accomplished.What does the poet mean by saying “the sad panorama of human woes/Passed daily under his eyes” in lines 63-64?The king looked down upon his miserable subjects.The king saw beggars passing by on the street. The king heard an overview of the sorrows of his people.The king was upset that his couriers found no happy men.In lines 65-66, how do the words “ashamed,” “useless,” and “maladies hatched in gloom” impact the tone of the poem?by ending the poem with a serious toneby continuing the poem’s foolish toneby supporting the poem’s apologetic toneby transitioning to a critical tone to end the poemPart A How does the poet develop the narrator’s point of view? by contrasting the narrator’s thoughts with the king’s thoughtsby stating the narrator’s objective observationsby using figurative language to convey the narrator’s attitudeby using the narrator’s inner dialogue to propel the plotPart B Which set of lines from the poem supports the answer in Part A? “But he said he was sick, and a king should know, And doctors came by the score.” (lines 5-6)“The old sage said, ‘You’re as sound as a nut.’ ‘Lock him up,’ roared the King in a gale, —” (lines 21-22)“The King will be well, if he sleeps one night In the Shirt of a Happy Man.” (lines 27-28)“‘I would do it, kind sir’ and he roared with the fun, ‘But I haven’t a shirt to my back.’” (lines 59-60)For this item, select the answer from the drop-down menu. Read the sentence from paragraph 4. This treacherous stretch of river would later be known for wrecking boats and taking lives of unskilled pilots. Use the drop-down menu to choose the word that best replaces the word treacherous in the sentence This freezingperilousscenicuncharted stretch of river would later be known for wrecking boats and taking lives of unskilled pilots. Which two statements show how the direct quotations in paragraphs 4 and 6 impact the passage? They reveal Emma Kelly’s adventurous and fearless nature.They convey the importance of safety during Emma Kelly’s trips through dangerous waters.They illustrate Emma Kelly’s desire to participate in activities from which women were historically excluded. They explain why Emma Kelly needed help in her journey to the Klondike.They describe the threats and hazards with which Emma Kelly and her partners dealt on a daily basis.Read the sentence from paragraph 10.The newspapers in Kansas and Missouri were all abuzz about her return.What does the author mean by the phrase “all abuzz”?using loud machineryunconcerned and lightheartedsharing exciting informationangered or annoyedPart A Which statement summarizes the central idea of the passage? With the backing of wealthy financiers, Emma Kelly moved to the Klondike to help set up a mining operation during the Gold Rush.Emma Kelly’s dog, Klondike, rescued her after she fell from a cliff by barking and jumping to alert the men in her traveling group.Emma Kelly’s bold and daring personality helped her find success during the Klondike Gold Rush as a journalist and mine owner.With her background knowledge and skill on rapid waters, Emma Kelly proved to be an asset to her traveling group as they made their way to the Klondike.Part BHow does the author convey the central idea selected in Part A?by providing historic and current statistics and data to show Emma Kelly’s profits as a business womanby using anecdotes from other members of the traveling group to reveal Emma Kelly’s impressive rescueby using detailed descriptions and firsthand accounts to illustrate Emma Kelly’s experiencesby providing a list of Emma Kelly’s accomplishments on her journey northBased on the information in the passage, what is the meaning of headstrong in paragraph 11 to describe Emma Kelly? dependent; anxioushealthy; toughirresponsible; hastywillful; determinedIn paragraph 11, what does the author mean by the figurative language “break the mold of society”?disobey rules and lawsleave one’s hometown on an adventurestart doing things in a new waytake dangerous risks despite being warnedRead the paragraph. Miles was having trouble with a math homework problem, so he asked his dad for help. His dad could not figure it out, so they found a math video online. Luckily, he explained clearly how to solve the problem. Which underlined pronoun has an unclear or ambiguous antecedent? “his”“it”“they”“he”A student is writing an essay on audiobooks, which are recordings of books being read aloud. Read the claim.  Listening to audiobooks helps children become strong readers. Which source is least likely to agree with the claim?  a nonprofit organization that creates free audiobooks of classic children’s literaturea principal who writes a blog called Back to the Basics: Learning to Read by Readinga salesperson who sells audiobook downloads to school librariesa technology company that makes an educational application called ListenReadRead the sentence. The musicians ___________ greeted audience members at the door. Which word fits best in the blank?  himselfitselfthemselvesyourselfChoose two sentences in which pronouns are used correctly. Her and I plan to wear matching shirts tomorrow.We decided to throw a surprise party for he and his friend.Omar forgot him backpack at home this morning.My dad’s car was parked outside with its trunk open.I cannot tell your sisters apart because they look so much alike.Read the sentence. Many things are forbidden on this beach; _______________________. Which can be placed after the semicolon to complete the sentence? campfires, glass bottles, and petsthe restrictions are unpopular with residentswhich makes it a frustrating place to visitbut the lifeguards rarely enforce the rulesA student is writing an essay on eating sushi. Read the paragraph. Eating sushi is a healthy choice. Sushi is a Japanese dish made with rice mixed with vinegar and a variety of natural ingredients, such as vegetables and uncooked fish. The components of sushi offer many health benefits. Omega-3 fatty acids, which are found in sushi and other seafood, play an important role in brain function. The student has found information from different sources. Which piece of information supports the claim in the paragraph? Sauces are often used to mask the true taste of the sushi, especially when the fish is not fresh.Sushi is usually served with sides such as soy sauce, wasabi, and gari, which is sweet, pickled ginger root.Nutritionists advise avoiding the “spicy” versions of sushi rolls, which usually contain mayonnaise.Nori, the seaweed often used to wrap sushi rolls, is high in key nutrients, such as vitamin A and iodine.A student is writing a research paper. Read the research question. Research Question: Are field trips worthwhile in middle school? Which source is least likely to have a strong bias?  a pamphlet published by the education department of an art museuma webpage titled “A Focus on Academics: How to Reduce Wasted Instructional Time”a study by an educational research group about the effects of field trips on student learninga newspaper article titled “Field Trip Gives Students Hands-on Learning Experiences”Part A A student is writing a research report on honey and maple syrup. The student has learned that the difference between honey and maple syrup is that honey is made by bees that collect nectar from flowering plants. Maple syrup, on the other hand, is made by humans who boil down the sap of maple trees. Read the student’s introductory sentence. Honey is very different from maple syrup, and it is made by a completely different process. Which sentence revision uses precise language to explain the difference between honey and maple syrup most effectively? Two types of sweeteners, honey and maple syrup, are made by plants and animals and can be used to add sweetness to foods.Honey and maple syrup can be used to replace granulated sugar in baking, and both offer health benefits when compared to sugar.Bees make honey from the nectar of flowering plants, but people make maple syrup from the sap of maple trees.Honey is sweeter than maple syrup, but both work well to sweeten cereal, baked goods, or granola.Part B A student is writing a research report on honey and maple syrup. The student has learned that the difference between honey and maple syrup is that honey is made by bees that collect nectar from flowering plants. Maple syrup, on the other hand, is made by humans who boil down the sap of maple trees. Read the student’s introductory sentence. Honey is very different from maple syrup, and it is made by a completely different process. Which is the most credible source for the answer to Part A? My Questions (myquestions.com)—This is a website that has general answers to questions about a wide range of topics including food.Charley’s Corner (charleyscorner.com)—This is a website created by a retired school teacher who wanted to blog about topics that interested him.Where Food Comes From (foodnutrition.gov)—This is a government website that gives information about common foods.Val’s Baking Project (valsbakingproject.net)—This is a website created by a student who tested recipes with different sweeteners.Choose the sentence in which commas should be added.The customers were so pleased with the service that they left extra money for a tip.The Biltmore House the largest privately owned house in the U.S. is located in North Carolina.The streets were filled with people cheering for the runners in the annual city marathon.The referees making the call were unsure if the player’s foot had stepped over the line.Read the paragraph. While Marcia practiced her clarinet, her father accompanied her on the piano. When she made a mistake, ___ would slow down, patiently waiting for her to catch up and find her place in the music. Which pronoun belongs in the blank? heherhistheirRead the paragraph. The anglerfish is one of many ocean animals to use bioluminescence, which is an organism’s ability to produce its own light. Anglerfish live on the ocean’s floor, so their glow lights up the otherwise lightless abyss. Only the female anglerfish uses bioluminescence; the glow comes from a fishing rod-like extension that protrudes from its forehead. Choose the most logical question based on evidence from the paragraph.  What advantages does its glowing extension provide to an anglerfish?When and how did scientists first discover the anglerfish?How many different species of anglerfish live in the ocean?Do scientists think that anglerfish will be affected by climate change?A student is writing a research paper on lobster. Read the evidence. Until the late 1800s, lobster was fed to prisoners as a way to save money. In fact, lobsters were often called the “cockroaches of the sea.” By World War II, lobster had earned a reputation as a food for the wealthy. It was featured at expensive restaurants in New York City and Boston. Today, lobster is among the most expensive protein sources on the market. A single pound of lobster meat can cost $50 or more. Based on the evidence gathered, choose the most logical research question the student can use to frame the inquiry.  How does the nutrition of lobster compare to that of other foods?How did lobster come to be considered a delicacy?What animals are the closest relatives to lobsters?What techniques do people use to catch lobsters?Read the paragraph. Megan wanted to keep her favorite shirt, but it was missing two buttons. Her grandfather stopped her from throwing it into the garbage can, assuring her that it could be fixed. _____ repaired the shirt together by sewing on two new buttons. Which pronoun belongs in the blank? HeItTheyWePart A Read the paragraph. Three-dimensional (3-D) televisions peaked in popularity between 2010 and 2013. Experts believe the main reason for the technology’s decline was its cost; 3-D televisions were more expensive than their two-dimensional counterparts, and the equipment needed to experience 3-D television, such as 3-D glasses and a 3-Denabled Blu-ray disc player, only added to the cost. Consumers simply were not willing to spend the extra money. ___________, most 3-D television sets and services are no longer available. Which transitional phrase best  fits in the blank?  As a resultFor instanceIn particularOn the contraryPart B Which text structure is indicated by the transitional phrase chosen in Part A? cause/effectclassificationcompare/contrastdefinitionRead the paragraph. Fred Rogers was the host and creator of the children’s television program Mister Rogers’ Neighborhood. A character on Rogers’ show, Officer Clemmons, was among the first African American characters on children’s television. In one episode, Mister Rogers and Officer Clemmons conversed while soaking their feet in a shared kiddie pool. At the time this episode aired in 1969, some people in the country were fighting to keep public swimming pools segregated by race. Based on the paragraph, what conclusion can be drawn about Fred Rogers? Fred Rogers’ gentle voice and kind spirit led to his success in children’s television.Fred Rogers used his television program to share his beliefs about the world.Fred Rogers’ television program aired on public television for 31 seasons.Fred Rogers disliked television advertising that targeted children.PauseSaved:08:13 amGraham, TheronID: 722136of45Sort by:Needs GradingNumerical OrderStandardsAnswers6-RI.5.11 ptQuestion 110620201011010Multiple Choice Question6-RI.8.11 ptQuestion 210620201021010Multiple Choice Question6-RI.5.11 ptQuestion 310620201041110Multiple Choice Question6-RI.9.11 ptQuestion 410620201052310Cloze Dropdown6-RI.8.21 ptQuestion 510620201061012Multiple Choice Question6-RI.9.11 ptQuestion 610620201071010Multiple Choice Question6-RI.6.11 ptQuestion 710620201101010Multiple Choice QuestionNS6-RI.11.11 ptQuestion 810620201111012Multiple Choice QuestionNS6-RI.11.21 ptQuestion 910620201131010Multiple Choice QuestionNS6-RL.10.61 ptQuestion 1010620202011010Multiple Choice QuestionNS6-RL.6.11 ptQuestion 1110620202022410Multiple Choice QuestionNS6-RL.5.11 ptQuestion 1210620202022710Multiple Choice QuestionNS6-RL.9.21 ptQuestion 1310620202041010Multiple Choice QuestionNS6-RL.12.11 ptQuestion 1410620202061010Multiple Choice QuestionNS6-RL.8.11 ptQuestion 1510620202101010Multiple Choice QuestionNS6-RL.11.11 ptQuestion 1610620202111010Multiple Choice QuestionNS6-RL.12.11 ptQuestion 1710620203071010Multiple Choice QuestionNS6-RL.9.11 ptQuestion 1810620203091010Multiple Choice QuestionNS6-RL.9.21 ptQuestion 1910620203081010Multiple Choice QuestionNS6-RL.11.11 ptQuestion 2010620203102410Multiple Choice QuestionNS6-RL.5.11 ptQuestion 2110620203102710Multiple Choice QuestionNS6-RI.9.11 ptQuestion 2210620204012310Cloze DropdownNS6-RI.8.21 ptQuestion 2310620204021110Multiple Choice QuestionNS6-RI.8.11 ptQuestion 2410620204031010Multiple Choice QuestionNS6-RI.6.11 ptQuestion 2510620204052410Multiple Choice QuestionNS6-RI.11.11 ptQuestion 2610620204052710Multiple Choice QuestionNS6-RI.9.11 ptQuestion 2710620204061010Multiple Choice QuestionNS6-RI.8.11 ptQuestion 2810620204041010Multiple Choice QuestionNS6-W.4.1.d1 ptQuestion 2910620200011010Multiple Choice QuestionNS6-I.3.31 ptQuestion 3010620200021010Multiple Choice QuestionNS6-W.4.11 ptQuestion 3110620200031010Multiple Choice QuestionNS6-W.4.1.a1 ptQuestion 3210620200041110Multiple Choice QuestionNS6-W.5.21 ptQuestion 3310620200051010Multiple Choice QuestionNS6-I.3.41 ptQuestion 3410620200061010Multiple Choice QuestionNS6-I.3.31 ptQuestion 3510620200081010Multiple Choice QuestionNS6-W.2.11 ptQuestion 3610620100082410Multiple Choice QuestionNS6-W.2.11 ptQuestion 3710620100082710Multiple Choice QuestionNS6-W.5.21 ptQuestion 3810620200091010Multiple Choice QuestionNS6-W.4.1.c1 ptQuestion 3910620200101010Multiple Choice QuestionNS6-I.2.11 ptQuestion 4010620200111010Multiple Choice QuestionNS6-I.1.11 ptQuestion 4110620200121010Multiple Choice QuestionNS6-W.4.1.c1 ptQuestion 4210620200141010Multiple Choice QuestionNS6-W.2.1.j1 ptQuestion 4310620200132410Multiple Choice QuestionNS6-W.2.1.c1 ptQuestion 4410620200132710Multiple Choice QuestionNS6-I.2.11 ptQuestion 4510620200151010Multiple Choice QuestionNSFinishedDiscardFinishedDiscardSubmitting Answers…Hang tight while we submit your answers.StudentYou’re Finished! DoneLaunching Respondus…Launching Respondus Your browser isn’t wide enough for the assessment.If you’re on a mobile device, try switching to landscape.OKCancel 000000

When we think of others as right-holders it is, of course, true that we no longer think of them as mere sub-jects, who plead abjectly for better treatment. We think of right-holdersas full persons, as citizens or citizens-to-be. Yet when claimants point toothers’ duties (which others’ duties?), they do not have to take muchaction, and may even wrap themselves passively in a cloak of grievanceor of resentment. They do not need to work out who will have to do whatfor whom at what cost, let alone what they themselves will have to do atwhat costs to themselves. In short, the rhetoric of rights, although moreactive than a rhetoric of dependent pleading, of mere subjects, is still arhetoric of recipience rather than of action. It still takes the perspectiveof the claimant rather than of the contributor, of the consumer ratherthan of the producer, of the passive rather than of the active citizen.

individually or collaboratively

Vannevar Bush July 1945 Issue

Annotation extends that power to a web made not only of linked resources, but also of linked segments within them. If the web is a loom on which applications are woven, then annotation increases the thread count of the fabric. Annotation-powered applications exploit the denser weave by defining segments and attaching data or behavior to them.

The idea of a system enabling a multiplicity of independent individuals to create lightweight value-added "trails" through a document space was envisaged most prominently by Vannevar Bush as early as 1945 [BUSH]. ComMentor can be thought of as a tool which such "trail blazers" use to add value to contents, and which other people use to find guidance based on these human-created super-structures. The overall architecture can be seen as a platform where value-added providers can provide their services (as a third player next to content providers and end users).

I'm not sure if it's blogging's fault, or journalism's fault or even Google's fault — but I do think the focus on recency as the biggest defining value of content is an error, and if we continue too far down that path, we'll regret it.

Finally, languages may also affect how individuals experience the world. According to the linguistic relativity hypothesis (Whorf, 1956), which is illustrated in Figure 3, individuals in an ethnolinguistic group are led by their shared language experiences to acquire shared, habitual ways of thinking, which influence cognition in a general way

First, her tippet made of tulle, easily lifted off her shoulders and laid on the back of a wooden chair.

Culture-sensitive healthcommunication will help to reduce disparities inhealth outcomes by making messages equally under-standable, meaningful, and effective.

This is not a new idea. It is based on the vision expounded by Vannevar Bush in his 1945 essay “As We May Think,” which conjured up a “memex” machine that would remember and connect information for us mere mortals. The concept was refined in the early 1960s by the Internet pioneer J. C. R. Licklider, who wrote a paper titled “Man-Computer Symbiosis,” and the computer designer Douglas Engelbart, who wrote “Augmenting Human Intellect.” They often found themselves in opposition to their colleagues, like Marvin Minsky and John McCarthy, who stressed the goal of pursuing artificial intelligence machines that left humans out of the loop.

how to approach the use of technology in learning environments to maximize opportunities for learning and minimize the risk of doing harm to students.

Powerful knowledge exerts its control not only over what it chooses to represent as real, but also in its repression of what it chooses to exclude.

They are happy, after all. The negro, situated as yours are, is not aware that he is deprived of any just rights." "Yes, yes," answered Georgiana: "you may place the slave where you please; you may dry up to your utmost the fountains of his feelings, the springs of his thought; you may yoke him to your labour, as an ox which liveth only to work, and worketh only to live; you may put him under any process which, without destroying his value as a slave, will debase and crush him as a rational being; you may do this, and the idea that he was born to be free will survive it all. It is allied to his hope of immortality; it is the ethereal part of his nature, which oppression cannot reach; it is a torch lit up in his soul by the hand of Deity, and never meant to be extinguished by the hand of man."         On reaching the drawing-room, they found Sam snuffing the candles, and looking as solemn and as dignified as if he had never sung a song or laughed in his life. "Will Miss Georgy have de supper got up now?" asked the negro. "Yes," she replied. "Well," remarked Carlton, "that beats anything I ever met with. Do you think that was Sam we heard singing?" "I am sure of it," was the answer. "I could not have believed that that fellow was capable of so much deception," continued Page 152 he. "Our system of slavery is one of deception; and Sam, you see, has only been a good scholar. However, he is as honest a fellow as you will find among the slave population here. If we would have them more honest, we should give them their liberty, and then the inducement to be dishonest would be gone. I have resolved that these creatures shall all be free." "Indeed!" exclaimed Carlton. "Yes, I shall let them all go free, and set an example to those about me." "I honour your judgment," said he. "But will the state permit them to remain?" "If not, they can go where they can live in freedom. I will not be unjust because the state is."

ome evidence suggests that suppression can even be beneficial for interdependent individuals. During a negative emotion induction, a stronger preference to regulate emotions led to a more adaptive pattern of physiological responding in Asian-American cultural contexts, while a stronger preference to regulate emotions led to a maladaptive pattern of physiological responding in European-American cultural contexts

The Buddha, who lived probably during the sixth centurybcin Nepal,began life as an aristocrat. After renouncing his palace and its haremand luxuries, he achieved through meditation an illumination in whichhe realized the great principle of the wheel of the law or the wheel of theBuddha. This may be defined as a theory of the “dependent origination”of life: that everything is conditioned by something else in a closed se-quence, so that in effect the misery of life is dependent upon certainconditions, and by eliminating these conditions it is possible to eliminatethe misery itself. Thus desire—which ultimately leads to misery—origi-nates in dependence upon sensation, which in turn originates in depen-dence upon contact and the six senses, and so on. The Buddhist objec-tive therefore becomes to cut the chain of conditions that bind one into

Young orphans need good upbringings, balanced diets,education, and technical training everywhere in the world; but unfortunately thereare many places where these goods are in such short supply that it is impossible toprovision all who need them. If we persist, nevertheless, in speaking of these needsas constituting rights and not merely claims, we are committed to the conception ofa right which is an entitlement to some good, but not a valid claim against anyparticular individual; for in conditions of scarcity there may be no determinateindividuals who can plausibly be said to have a duty to provide the missing goodsto those in need. J. E. S. Fawcett therefore prefers to keep the distinction betweenclaims, and rights firmly in mind. "Claims," he writes, "are needs and demands inmovement, and there is a continuous transformation, as a society advances [towardsgreater abundance] of economic and social claims into civil and political rights.. .and not all countries or all claims are by any means at the same stage in theprocess."8 The manifesto writers on the other side who seem to identify needs, orat least basic needs, with what they call "human rights," are more properly de-scribed, I think, as urging upon the world community the moral principle that allbasic human needs ought to be recognized as claims (in the customary prima faciesense) worthy of sympathy and serious consideration right now, even though, inmany cases, they cannot yet plausibly be treated as valid cla.ims, that is, as groundsof any other people's duties. This way of talking avoids the anomaly of ascribingto all human beings now, even those in pre-industrial societies, such "economicand social rights" as "periodic holidays with pay."9

We point out that dark-energy perturbations may become unstable in the presence ofa gravitational wave of sufficiently large amplitude. We study this effect for the cubicHorndeski operator (braiding), proportional toαB.

Rising CO2’s effect on crops could also harm human health. “We know unequivocally that when you grow food at elevated CO2 levels in fields, it becomes less nutritious,” notes Samuel Myers, principal research scientist in environmental health at Harvard University. “[Food crops] lose significant amounts of iron and zinc—and grains [also] lose protein.” Myers and other researchers have found atmospheric CO2 levels predicted for mid-century—around 550 parts per million—could make food crops lose enough of those key nutrients to cause a protein deficiency in an estimated 150 million people and a zinc deficit in an additional 150 million to 200 million. (Both of those figures are in addition to the number of people who already have such a shortfall.) A total of 1.4 billion women of child-bearing age and young children who live in countries with a high prevalence of anemia would lose more than 3.8 percent of their dietary iron at such CO2 levels, according to Meyers.dfp.loadAds("right2","MPU3","dfp-right2-article-2")Advertisement Researchers do not yet know why higher atmospheric CO2 alters crops’ nutritional content. But, Myers says, “the bottom line is, we know that rising CO2 reduces the concentration of critical nutrients around the world,” adding that these kinds of nutritional deficiencies are already significant public health threats, and will only worsen as CO2 levels go up.

Rooted and Roadblocked

s I say, if we bring up religion, we’ll have differences, we’ll have arguments, and we’ll never be able to get together

“Rise and shine, my dear!” Patricia exclaimed.There was no movement in the bed.“Honey, are you alright?”Moans of sickness reach Patricia’s ears.“Oh no, this doesn’t sound good. Susan, how are you feeling? What hurts?”“I feel sick, like I can’t get up. Where am I?”“Oh no honey, okay. I will be right back.” Patricia ran out of the room and dialed 911. She explained the emergency to the operator, and they said they would send an ambulance for Susan right away. Susan would have to go to the hospital by herself, as human travel was only for those experiencing the emergency. Patricia then found Timothy and explained her symptoms to Timothy. “It seems like she has carbon monoxide poisoning, but I have no idea how she would have gotten it if she did not go outside.”The concerned parents then rushed into Susan’s room to monitor her.“The ambulance is on their way, Susan, everything is going to be okay.” Timothy assured Susan.Susan moaned to signal she understood and reached for her father’s hand.“Well, you were correct in your diagnosis of carbon monoxide poisoning. Miss Martin had high levels of carboxyhemoglobin in her blood. We are currently giving her pure oxygen to breathe in to increase oxygen levels in the blood and help to remove carbon monoxide from the blood. The concentration in her blood was not high enough to kill her, but it was elevated enough to produce undesirable effects to her health. We will keep her on the oxygen for at leave four hours and reevaluate her condition then. We will call you with the update at 1 o’clock.” Susan’s doctor explained on the phone with Patricia.“Thank you, Doctor, we will eagerly await your phone call. Talk with you soon, bye.” Patricia replied.“Well, Dear, how did this happen? We know that carbon monoxide fills the air outside, but our home should be sealed shut and oxygen should be pumping into our home.” Timothy stated.“I really don’t know, Tim. We will have to do some investigating. Let’s check out her room.” Patricia replied.Timothy and Patricia then moved from where they had been sitting since Susan left for the hospital that morning to Susan’s room.“Well, if this room has a leak somewhere, you would think that we would have some symptoms too.” Timothy suggested.“You’re right, Tim. But maybe there is just a low concentration of it in our blood now because we haven’t been in her room as long as she had, so we do not have any symptoms yet.” Patricia proposed.“Very true, Patricia. If that is so, where would the leak be?”They began to search the room for any suspicious or damaged aspects of the room. They searched up and down, left and right. Nothing could be found. They resorted to moving Susan’s bed and looking behind all other furniture just to be sure. Again, nothing.“You don’t think she did, do you?” Patricia asked timidly.“What? Oh, no, no honey, Susan would never.” Timothy assured her.Patricia was wondering if Susan may have left the house for a period of time, perhaps long enough to poison her. The possibility seemed impossible, so they disregarded it and agreed that carbon monoxide had to be leaking into Susan’s room. Therefore, with no source of leakage found, they took measures to seal off their own daughter’s bedroom so that no carbon monoxide would contaminate the oxygen that filled the remainder of their home. 1 o’clock would be there sooner than they knew, so they decided to eat some lunch first to get some food on their stomachs.

such as a student’s sleeping environment and de-stressing techniques if the student has troubles falling asleep. It would help the aspiring student become aware of more specific ways they may be able to utilize their sleep habits in the hopes of improving their grades.

True. But as they say Dreams are rough copies of the waking soul Yet uncorrected of the higher Will, So that men sometimes in their dreams confess An unsuspected, or forgotten, self; One must beware to check—ay, if one may, Stifle ere born, such passion in ourselves As makes, we see, such havoc with our sleep, And ill reacts upon the waking day. And, by the bye, for one test, Segismund, Between such swearable realities— Since Dreaming, Madness, Passion, are akin In missing each that salutary rein Of reason, and the guiding will of man: One test, I think, of waking sanity Shall be that conscious power of self-control, To curb all passion, but much most of all That evil and vindictive, that ill squares With human, and with holy canon less, Which bids us pardon ev’n our enemies, And much more those who, out of no ill will, Mistakenly have taken up the rod Which heaven, they think, has put into their hands.

Food production systems such as organic agriculture and grass-fed beef have been proposed as potential ways to reduce agriculture's environmental impacts (e.g. Ponisio et al 2014). Organic agriculture, for example, is often promoted as having lower environmental impacts relative to high-input conventional systems because it replaces agrochemical inputs with natural inputs such as manure or with ecosystem services such as pest control (Azadi et al 2011).

In the United States, women typically score somewhat higher than men on both N and A, as well as some specific facets of E and O (e.g., Warmth, Openness to Aesthetics). Men usually score higher on other facets of E and O, namely, Assertiveness and Openness to Ideas. There are few gender differences in C.

We may as well assert that because a child has thrived upon milk, that it is never to have meat, or that the first twenty years of our lives is to become a precedent for the next twenty.

. Itscloutresidesinidea,not inforce.

We may know, for example, that television exaggerates the scale of violent crime for dramatic purposes; nevertheless, studies show that the more television we watch, the more violent we assume the world to be.

While Allen’s work is largely framed as a way todirect the meaningful support of educators to encourage the civic participa-tion of young people, we instead take student letters as evidence of, ratherthan an intervention on behalf of, youth’s verbal empowerment, democraticknowledge, and participatory readiness.

7-Eleven CDO And CIO Creates Experiences Of The Future For Customers Peter High Contributor Opinions expressed by Forbes Contributors are their own. Enterprise & Cloud Share to facebook Share to twitter Share to linkedin 7-Eleven is a big retailer. It operates 65,000 stores in 18 countries, has 55 million customers in stores on a daily basis, and conducts 20 billion transactions annually. Like most retailers, it is in need of transformation. Enter Gurmeet Singh. With a Ph.D. in Engineering from Rice, and stints with leading companies like FedEx, Intuit, and Capital One, Singh joined 7-Eleven in August of 2016 as the chief digital officer. He would add the chief information officer title in November 2017.  Singh joined the company with a mandate from 7-Eleven CEO Joseph DePinto to make the company a digital leader. Singh embarked on a multi-year journey to become a digitally-enabled organization, including a “full stack transformation” approach which encompasses consumer-facing technology, back-end technology, infrastructure, and the organizational stack. He also expanded the company's loyalty program from its initial focus on beverages to a full-fledged loyalty offering that is available on mobile, web, digital loyalty card, and even through chatbots. Singh notes that "the closest store to a customer today is in the palm of [his or her] hand," and he wanted to go beyond pushing customers from mobile into stores and allowing customers to interact with 7-Eleven on their terms, through the interface of their choice. To foster this, Singh's team helps the rest of the organization understand the art of the possible through constant experimentation with new technology. He describes his path to innovation in great detail herein.  (To listen to an unabridged audio version of this interview, please visit this link. This is the 38th interview in the CIO-plus series. To read the prior interviews with CIO-pluses from companies such as Intel, P&G, Biogen, Kroger, and Cardinal Health, please click this link. To read future articles in this series, please follow me on Twitter @PeterAHigh.) Peter High: You are the Chief Digital and Chief Information Officer of 7-Eleven. Could you describe your role? Today In: Tech Gurmeet Singh, Chief Digital and Chief Information Officer of 7-Eleven Credit: 7-Eleven Singh: I started at 7-Eleven as the Chief Digital Officer, with my primary responsibility being driving digital transformation. This company founded convenience at a global scale, with 65,000 stores in 18 countries, 55 million customers visiting the stores every day, and 20 billion transactions on an annual basis. You take that, and you overlay the consumer trends and the new technology trends like big data and digital payments, and you have the perfect formula for redefining convenience. PROMOTED Forbes Insights Deloitte BrandVoice  | Paid Program When It Comes To Readiness, The Fundamentals Still Matter .str-more-from-forbes .thumbnail-wrapper{margin:0 !important;}.str-more-from-forbes .str-opt-out{height:12px !important;width:12px !important;background-size:12px 12px !important;min-height:12px !important;max-height:12px !important;min-width:12px !important;max-width:12px !important;}.str-more-from-forbes.str-editorial-true .str-voice{display:none !important;}.str-more-from-forbes.str-insights-true .str-voice{display:none !important;}.str-more-from-forbes.str-insights-true .str-insights{display:inline-block !important;}.str-more-from-forbes h3.headline{font-size:0.7em;line-height:1.6em;color:#333;letter-spacing:0;margin:0.3em 0;color:#333;letter-spacing:0;}.str-more-from-forbes .str-bv{font-size:.9em;letter-spacing:0;margin-bottom:2px;} (function(){ var el = document.getElementsByClassName("ntv-ad")[0]; el.style.display = 'block'; })(); Forbes Insights Forbes BrandVoice  | Paid Program The Vizio D Is On Sale For $480 At Walmart .str-more-from-forbes .thumbnail-wrapper{margin:0 !important;}.str-more-from-forbes .str-opt-out{height:12px !important;width:12px !important;background-size:12px 12px !important;min-height:12px !important;max-height:12px !important;min-width:12px !important;max-width:12px !important;}.str-more-from-forbes.str-editorial-true .str-voice{display:none !important;}.str-more-from-forbes.str-insights-true .str-voice{display:none !important;}.str-more-from-forbes.str-insights-true .str-insights{display:inline-block !important;}.str-more-from-forbes h3.headline{font-size:0.7em;line-height:1.6em;color:#333;letter-spacing:0;margin:0.3em 0;color:#333;letter-spacing:0;}.str-more-from-forbes .str-bv{font-size:.9em;letter-spacing:0;margin-bottom:2px;} (function(){ var el = document.getElementsByClassName("ntv-ad")[0]; el.style.display = 'block'; })(); Forbes Insights Grads of Life BrandVoice  | Paid Program More Than What’s On Paper: Dedication Has Its Rewards .str-more-from-forbes .thumbnail-wrapper{margin:0 !important;}.str-more-from-forbes .str-opt-out{height:12px !important;width:12px !important;background-size:12px 12px !important;min-height:12px !important;max-height:12px !important;min-width:12px !important;max-width:12px !important;}.str-more-from-forbes.str-editorial-true .str-voice{display:none !important;}.str-more-from-forbes.str-insights-true .str-voice{display:none !important;}.str-more-from-forbes.str-insights-true .str-insights{display:inline-block !important;}.str-more-from-forbes h3.headline{font-size:0.7em;line-height:1.6em;color:#333;letter-spacing:0;margin:0.3em 0;color:#333;letter-spacing:0;}.str-more-from-forbes .str-bv{font-size:.9em;letter-spacing:0;margin-bottom:2px;} (function(){ var el = document.getElementsByClassName("ntv-ad")[0]; el.style.display = 'block'; })(); My role as the Chief Digital Officer is to make this happen for our customers, our franchisees, our store associates, and our employees. One of our strategic pillars is to become a digitally-enabled organization. In my role as the CDO, I set the strategy and the vision. We are developing new business models and new digital product experiences, which also includes establishing a product management discipline, user experience design, data scientists, digital marketers, and digital technologists. Technology is a key element of a digital transformation and it is also key to becoming a digitally-enabled company. Initially, we started off with what most companies have been doing and what most consulting companies have been citing as a strategic approach, which is building a two-tier architectural model. A two-tier architectural model means you have digital technology capabilities that are being developed at a higher speed, and then decoupled from that, you have your legacy enterprise systems which have longer release cycles at slower speeds. Additionally, enterprise and legacy work was being managed in long cycle processes as projects, not as products. Old models in any company are always changing, and they should be changing. They are a function of maturity of the company, the need of the hour, and the market factor. As we evaluated the speed of our transformation, we felt that we were not getting to the speed we needed. To get there, we needed what I call a full stack transformation. When I say full stack transformation, I am talking all the way from the consumer-facing technology to back-end technologies, all the way to infrastructure and cloud. It even goes beyond that to encompass the organizational stack. To bring more efficiency and effectiveness to our decisions, our prioritization, while driving the productization of IT, we decided to combine the CDO role and the CIO role. This allows us to drive vertical product slices while working on horizontal capabilities. If you are doing one after the other, you are taking too much time to get the business transformed. If you do not do a vertical slice, you do not know what customer problem you are trying to solve. Combining the functions gets us there faster. It is harder, but you end up driving more synergies. We drive higher team engagement. You speed up your transformation journey, and you end up creating a stronger pool of talent as one team. What we did was then combine digital and IT, which we call DIGIT, which is very much digital. High: You are clearly thinking multiple years out. The changes require hard work to be done in the near term to make the organization nimbler for the long term. How difficult was the process of selling this internally? Was it difficult in having your peers among the executive leadership understand the rationale behind all the hard work to be done in the near term for a better outcome for the long term? Singh: It was not that difficult, which is the great part of being at 7-Eleven. Our entire executive team meets regularly, and we often share our progress with each other. The key here was Joe DePinto, our CEO, and his vision and his desire to become a digitally-enabled organization and make it a strategic pillar. Joe is fully supportive of our full stack transformation. Once we realized that the change must be beyond our borders and immediate sphere of influence, it did not take long to gain executive buy-in. For us to move fast and drive the transformation we need, we can do both things together. It is all about talent. If we can get the right talent and the right people, we can make it happen. High: With so much technology change, people often focus on the technology. However, it is the people that are ultimately needed to drive that change. The skills necessary for this type of transformation are in great demand, and I am wondering how you attract top talent. How do you get top technical talent to come work for 7-Eleven? Singh: The talent needed to drive this kind of transformation is in high demand. However, based on my background at FedEx, Intuit, and Capital One, I am able to tell these talented individuals my story and show them what I can achieve at scale. Additionally, we have a great story and it is an incredible opportunity for anyone seeking out a problem who wants to be part of the solution. This story helped sell me on the organization, as I saw the company was sitting on such a strong infrastructure, and I knew we could do great things if we combined it with digital. Up until now, the success of 7-Eleven has been based on a strong operations team and a strong merchandising team. The success of the future is operations, plus merchandising, plus geeks. That is why we are looking for talent. We have been able to attract great talent, and as you start building momentum, it gets easier to attract more talent. That is why it is critical for us to go out and build stuff in the market and share the success. We want people to know what a huge opportunity we have. High: You have also set a goal to, "Redefine convenience with software." Can you describe what you mean by that and the methods of doing so? Singh: We have been a global leader in convenience, but just being physically convenient and having a store physically available, conveniently accessible, and open 24/7 is not enough for the future. The closest store to a customer today is in the palm of their hand. It is in their voices assistant, or their connected car, or their connected home. That is why I believe most retailers have struggled. Most of them do not think out of the box. They focus on creating the store of the future and driving traffic to the stores. At 7-Eleven we are focused on creating what I call “experiences of the future,” which are convenient and delightful. Our stores play an important role, but they are not the whole equation. When I talk about redefining convenience, I am talking about how we can be in the consumer's ecosystem of mobile devices and voice assistants and maps. We want to be wherever our customer is spending their time. When we bring the store to them, how will we bring it to them? We will bring it to them through partnerships, through delivery, and through APIs. To create these delightful experiences and in-store experiences, we require technology. We need to be acting like a software company, an API company, a big data company, and truly become a technology first company. For example, Airbnb, Amazon, and Uber are software companies at their core. Uber redefined convenience in the taxi business through software and ended up creating a new market. Another key element to redefining convenience with software is talent. We need to bring in the product management mindset to how we work, how we are agile, the design thinking, the technology to solve customer problems, and create a maker culture. High: You are also rethinking loyalty. You have the Seven Rewards programs. Could you talk about the evolution of that and the role that it plays in combination with some of the themes you have already discussed? Singh: At 7-Eleven, we want every customer interaction to be valuable and delightful. We can do this by building personalized connections with our customers. Loyalty becomes a key element of building those personalized connections. We have expanded our already successful loyalty program which is focused on what we call the cups program. When customers would buy six cups of our proprietary beverages, they will get the seventh one free. We have extended that in many ways. We give customers more ways to participate in the program. Before, it was only through the mobile app. We have since added mobile web, we added a loyalty card, and we added a digital loyalty card through our Facebook Messenger chatbot. That gives our customers more ways to engage with the program. We also give them more ways to earn and drive more engagement. Customers can earn points on most items they purchase and get bonus points. It creates this gamification aspect because they can earn points here and then they can earn more points when they buy more. It gets our customers more engaged with our program. They can choose to redeem these points through a menu of products in the store, which is something we have expanded as well. Think about the program going from only a cups program to more ways to engage, more ways to join, more ways to earn, and more ways to get value out of the program. We have 12 million users of our app, and our App Store ratings have increased substantially. We went from an iOS rating of 1.5 to 4.8. On the food and drink category in the App Store, we were as high as number four. If you look across all apps, we were at number 57 very recently. That speaks to the new user experience we provided in the app and all the new capabilities we are offering our customers. High: Customers have evolved to the point where they now expect mobile and digital technology to complement the physical experiences of traditional retail. Can you talk about the methods that you use to keep up with these evolving desires? Singh: Being usable is not good enough. You must create experiences that delight the customer and drive emotion. Users are expecting things to just happen for them. Customers expect recommendations, frictionless experiences, and for things to just happen. You have to put yourself in the customer’s shoes and be conscious of the customer journey. This includes not just the journey of today’s customer but the potential journey of the customer tomorrow. Additionally, being able to provide the customer more than what they expected, what I like to call innovating the customer, is critical. An example I like to use is the combination of user interfaces and new technologies. Specifically, voice search and voice technology is something I find quite interesting because it can really drive new user experiences. Data tells us that 50% of all future searches will be voice searches. This is an example of a frictionless experience because while a typical person can only type 40 words per minute, they can speak 150 per minute, which drives the new user experience. Technology has become part of the equation, but the number one thing is we want to look at what the journey is, where the pain points are, and what problems we are trying to solve. All that goes back to the methodology of design thinking, journey mapping, and the whole product management mindset. We do A/B testing and build minimal viable products. Doing this continuous learning helps you evolve. These are the types of combinations we are looking at when we think about the customer journey and friction points. We are focused on delighting them and anticipating what they want even if they did not ask for it. First, we think through the experience, and then we think about what technologies can drive that experience. These are the technologies we continuously test and make improvements on. High: You mentioned machine learning and voice-enabled technology. These are a couple of examples of a broader panoply of emerging technologies that you and your team are experimenting with. Could you talk about your thought process on engaging with some of these newer technologies? Do you have a portion of your team that is focused on the art of the possible or an innovation lab? Singh: To become a full stack technology organization and provide these experiences to the customers, a lot of the back-end technologies have to move from batch oriented to real time. Hence, the combination of transformation cannot be only on one layer. It must cut across all the layers. We are putting together an R&D center, but we already test new technologies. We have a CTO function that reports to me. Part of the purview of that CTO function is to test new technologies and conduct proof of concepts. The way we think about them is in two categories. One is technologies that are already proven and are ready to scale. Number two is what we call emerging technologies. We are building capabilities with production-ready technologies even beyond things like serverless microservices-based architectures, chatbots, big data, and AI. These are areas where we are looking to scale as we drive the business benefits back to our customers, back to our store associates, back to our franchisees, and even back to our internal stakeholders like merchandising and operations. For the emerging technology, what we do is closely watch and actively experiment. Today, a lot of software libraries are open-sourced, such as TensorFlow from Google. They allow others to experiment because they want companies creating proof of concepts (POCs) with them. Facebook Messenger is one example where we leveraged the chatbot for loyalty signup. That has not been done before. We were able to take the physical card and convert that into a digital card on Facebook Messenger. We are continuously trying to experiment with technologies like blockchain, and we are building POCs to validate and test in the market. We are going to formalize a full R&D group into an innovation lab, but while that is going on, that is not stopping us from testing these things. From an emerging technology and testing approach, the fact that we have a global scale and presence is a huge benefit. Not long ago, I wrote a blog post on LinkedIn titled “Thinking Glocally” which means think globally and act locally. The customer trends and technology trends that we are encountering today are global. I have visited Korea and China, and I see that all these countries are facing the same customer and technology trends. Of course, they may be at different stages of maturity, and some are leaping ahead. When I was in China, I saw the massive adoption of digital payments. I was surprised that the US is so behind with digital payments compared to what I saw in China, which is primarily driven by WeChat and AliPay. When I visited our South Korea team, they had introduced an interesting concept of biometrics and unmanned stores with pay-by-hand. What I learned was the veins in your palm are better for biometric authentication than your fingerprints. They are more secure and more sanitary. You end up registering your palm, link it to a payment method, and off you go. Our stores in Thailand just announced the launch of facial recognition. Beyond our labs, these stores give us a presence in the world and provide an opportunity to test. We do not have to prioritize them here, we can test them outside, bring them back, and vice versa. That has helped us in multiple ways on experimenting with emerging technologies and testing new concepts. All of this is in service to the customer experience and the store experience. Peter High is President of Metis Strategy, a business and IT advisory firm. His latest book is Implementing World Class IT Strategy. He is also the author of World Class IT: Why Businesses Succeed When IT Triumphs. Peter moderates the Forum on World Class IT podcast series. He speaks at conferences around the world. Follow him on Twitter @PeterAHigh.

Global emissions are still rising, hitting a record last year. Meanwhile, the world’s population is growing.

Many climate scientists think the IPCC’s backward-looking, consensus-based estimates are too optimistic,

The practical things in religion are those whose universals are in practical philosophy. That is because the practical things in religion are those universals made determinate by stipulations restricting them, and what is restricted by stipulations is more particular than what is pronounced unqualifiedly without stipulations: for instance, our saying “the human being who is writing” is more particular than our saying “the human being.” Therefore, all virtuous laws are subordinate to the universals of practical philosophy. The theoretical opinions that are in religion have their demonstrative proofs in theoretical philosophy and are taken in religion without demonstrative proofs.

As the all is larger than the part double the large which was considered as absolute /would be larger than the large which is considered as the absolute large. However by “the absolute large”  is meant simply that than which nothing is larger, and therefore the absolute large would not be an absolute large. Either, then, there will be no (absolute) large whatsoever, or there will be a relatively large since the large is not predicated other than absolutely or relatively.        /If the absolute large were not large absolutely) it would be an existant non-existent and this is an impossible contradiction; while if the absolute large were the relatively large, absolute and relative would be synonymous terms for the same thing, vis., that another thing is smaller than it; since it has been explained that there can never bee a thing which does not have something larger than it, either in potentiality or in actuality.        In this manner, it may be explained that there cannot be an absolute small and that the small also occurs only relatively.While the large and small are predicated of all quantitative things, the long and short are predicated of all quantitative things which are continuous, and they are specific to the continuous and not to the other kinds of quantity. /They also are predicated relatively only, and not as an absolute predicate

We live in an age of paradox. Systems using artificial intelligence match or surpass human level performance in more and more domains, leveraging rapid advances in other technologies and driving soaring stock prices. Yet measured productivity growth has fallen in half over the past decade, and real income has stagnated since the late 1990s for a majority of Americans. Brynjolfsson, Rock, and Syverson describe four potential explanations for this clash of expectations and statistics: false hopes, mismeasurement, redistribution, and implementation lags. While a case can be made for each explanation, the researchers argue that lags are likely to be the biggest reason for paradox. The most impressive capabilities of AI, particularly those based on machine learning, have not yet diffused widely. More importantly, like other general purpose technologies, their full effects won't be realized until waves of complementary innovations are developed and implemented. The adjustment costs, organizational changes and new skills needed for successful AI can be modeled as a kind of intangible capital. A portion of the value of this intangible capital is already reflected in the market value of firms. However, most national statistics will fail to capture the full benefits of the new technologies and some may even have the wrong sign

Adolescents in different cultures spend different amount of time with their peers

As a prototype it hits a sweet spot: it's challenging - it's no small feat to recognize handwritten digits - but it's not so difficult as to require an extremely complicated solution, or tremendous computational power. Furthermore, it's a great way to develop more advanced techniques, such as deep learning. And so throughout the book we'll return repeatedly to the problem of handwriting recognition. Later in the book, we'll discuss how these ideas may be applied to other problems in computer vision, and also in speech, natural language processing, and other domains.Of course, if the point of the chapter was only to write a computer program to recognize handwritten digits, then the chapter would be much shorter! But along the way we'll develop many key ideas about neural networks, including two important types of artificial neuron (the perceptron and the sigmoid neuron), and the standard learning algorithm for neural networks, known as stochastic gradient descent. Throughout, I focus on explaining why things are done the way they are, and on building your neural networks intuition. That requires a lengthier discussion than if I just presented the basic mechanics of what's going on, but it's worth it for the deeper understanding you'll attain. Amongst the payoffs, by the end of the chapter we'll be in position to understand what deep learning is, and why it matters.PerceptronsWhat is a neural network? To get started, I'll explain a type of artificial neuron called a perceptron. Perceptrons were developed in the 1950s and 1960s by the scientist Frank Rosenblatt, inspired by earlier work by Warren McCulloch and Walter Pitts. Today, it's more common to use other models of artificial neurons - in this book, and in much modern work on neural networks, the main neuron model used is one called the sigmoid neuron. We'll get to sigmoid neurons shortly. But to understand why sigmoid neurons are defined the way they are, it's worth taking the time to first understand perceptrons.So how do perceptrons work? A perceptron takes several binary inputs, x1,x2,…x1,x2,…x_1, x_2, \ldots, and produces a single binary output: In the example shown the perceptron has three inputs, x1,x2,x3x1,x2,x3x_1, x_2, x_3. In general it could have more or fewer inputs. Rosenblatt proposed a simple rule to compute the output. He introduced weights, w1,w2,…w1,w2,…w_1,w_2,\ldots, real numbers expressing the importance of the respective inputs to the output. The neuron's output, 000 or 111, is determined by whether the weighted sum ∑jwjxj∑jwjxj\sum_j w_j x_j is less than or greater than some threshold value. Just like the weights, the threshold is a real number which is a parameter of the neuron. To put it in more precise algebraic terms: output={01if ∑jwjxj≤ thresholdif ∑jwjxj> threshold(1)(1)output={0if ∑jwjxj≤ threshold1if ∑jwjxj> threshold\begin{eqnarray} \mbox{output} & = & \left\{ \begin{array}{ll} 0 & \mbox{if } \sum_j w_j x_j \leq \mbox{ threshold} \\ 1 & \mbox{if } \sum_j w_j x_j > \mbox{ threshold} \end{array} \right. \tag{1}\end{eqnarray} That's all there is to how a perceptron works!That's the basic mathematical model. A way you can think about the perceptron is that it's a device that makes decisions by weighing up evidence. Let me give an example. It's not a very realistic example, but it's easy to understand, and we'll soon get to more realistic examples. Suppose the weekend is coming up, and you've heard that there's going to be a cheese festival in your city. You like cheese, and are trying to decide whether or not to go to the festival. You might make your decision by weighing up three factors: Is the weather good? Does your boyfriend or girlfriend want to accompany you? Is the festival near public transit? (You don't own a car). We can represent these three factors by corresponding binary variables x1,x2x1,x2x_1, x_2, and x3x3x_3. For instance, we'd have x1=1x1=1x_1 = 1 if the weather is good, and x1=0x1=0x_1 = 0 if the weather is bad. Similarly, x2=1x2=1x_2 = 1 if your boyfriend or girlfriend wants to go, and x2=0x2=0x_2 = 0 if not. And similarly again for x3x3x_3 and public transit.Now, suppose you absolutely adore cheese, so much so that you're happy to go to the festival even if your boyfriend or girlfriend is uninterested and the festival is hard to get to. But perhaps you really loathe bad weather, and there's no way you'd go to the festival if the weather is bad. You can use perceptrons to model this kind of decision-making. One way to do this is to choose a weight w1=6w1=6w_1 = 6 for the weather, and w2=2w2=2w_2 = 2 and w3=2w3=2w_3 = 2 for the other conditions. The larger value of w1w1w_1 indicates that the weather matters a lot to you, much more than whether your boyfriend or girlfriend joins you, or the nearness of public transit. Finally, suppose you choose a threshold of 555 for the perceptron. With these choices, the perceptron implements the desired decision-making model, outputting 111 whenever the weather is good, and 000 whenever the weather is bad. It makes no difference to the output whether your boyfriend or girlfriend wants to go, or whether public transit is nearby.By varying the weights and the threshold, we can get different models of decision-making. For example, suppose we instead chose a threshold of 333. Then the perceptron would decide that you should go to the festival whenever the weather was good or when both the festival was near public transit and your boyfriend or girlfriend was willing to join you. In other words, it'd be a different model of decision-making. Dropping the threshold means you're more willing to go to the festival.Obviously, the perceptron isn't a complete model of human decision-making! But what the example illustrates is how a perceptron can weigh up different kinds of evidence in order to make decisions. And it should seem plausible that a complex network of perceptrons could make quite subtle decisions: In this network, the first column of perceptrons - what we'll call the first layer of perceptrons - is making three very simple decisions, by weighing the input evidence. What about the perceptrons in the second layer? Each of those perceptrons is making a decision by weighing up the results from the first layer of decision-making. In this way a perceptron in the second layer can make a decision at a more complex and more abstract level than perceptrons in the first layer. And even more complex decisions can be made by the perceptron in the third layer. In this way, a many-layer network of perceptrons can engage in sophisticated decision making.Incidentally, when I defined perceptrons I said that a perceptron has just a single output. In the network above the perceptrons look like they have multiple outputs. In fact, they're still single output. The multiple output arrows are merely a useful way of indicating that the output from a perceptron is being used as the input to several other perceptrons. It's less unwieldy than drawing a single output line which then splits.Let's simplify the way we describe perceptrons. The condition ∑jwjxj>threshold∑jwjxj>threshold\sum_j w_j x_j > \mbox{threshold} is cumbersome, and we can make two notational changes to simplify it. The first change is to write ∑jwjxj∑jwjxj\sum_j w_j x_j as a dot product, w⋅x≡∑jwjxjw⋅x≡∑jwjxjw \cdot x \equiv \sum_j w_j x_j, where www and xxx are vectors whose components are the weights and inputs, respectively. The second change is to move the threshold to the other side of the inequality, and to replace it by what's known as the perceptron's bias, b≡−thresholdb≡−thresholdb \equiv -\mbox{threshold}. Using the bias instead of the threshold, the perceptron rule can be rewritten: output={01if w⋅x+b≤0if w⋅x+b>0(2)(2)output={0if w⋅x+b≤01if w⋅x+b>0\begin{eqnarray} \mbox{output} = \left\{ \begin{array}{ll} 0 & \mbox{if } w\cdot x + b \leq 0 \\ 1 & \mbox{if } w\cdot x + b > 0 \end{array} \right. \tag{2}\end{eqnarray} You can think of the bias as a measure of how easy it is to get the perceptron to output a 111. Or to put it in more biological terms, the bias is a measure of how easy it is to get the perceptron to fire. For a perceptron with a really big bias, it's extremely easy for the perceptron to output a 111. But if the bias is very negative, then it's difficult for the perceptron to output a 111. Obviously, introducing the bias is only a small change in how we describe perceptrons, but we'll see later that it leads to further notational simplifications. Because of this, in the remainder of the book we won't use the threshold, we'll always use the bias.I've described perceptrons as a method for weighing evidence to make decisions. Another way perceptrons can be used is to compute the elementary logical functions we usually think of as underlying computation, functions such as AND, OR, and NAND. For example, suppose we have a perceptron with two inputs, each with weight −2−2-2, and an overall bias of 333. Here's our perceptron: Then we see that input 000000 produces output 111, since (−2)∗0+(−2)∗0+3=3(−2)∗0+(−2)∗0+3=3(-2)*0+(-2)*0+3 = 3 is positive. Here, I've introduced the ∗∗* symbol to make the multiplications explicit. Similar calculations show that the inputs 010101 and 101010 produce output 111. But the input 111111 produces output 000, since (−2)∗1+(−2)∗1+3=−1(−2)∗1+(−2)∗1+3=−1(-2)*1+(-2)*1+3 = -1 is negative. And so our perceptron implements a NAND gate!The NAND example shows that we can use perceptrons to compute simple logical functions. In fact, we can use networks of perceptrons to compute any logical function at all. The reason is that the NAND gate is universal for computation, that is, we can build any computation up out of NAND gates. For example, we can use NAND gates to build a circuit which adds two bits, x1x1x_1 and x2x2x_2. This requires computing the bitwise sum, x1⊕x2x1⊕x2x_1 \oplus x_2, as well as a carry bit which is set to 111 when both x1x1x_1 and x2x2x_2 are 111, i.e., the carry bit is just the bitwise product x1x2x1x2x_1 x_2: To get an equivalent network of perceptrons we replace all the NAND gates by perceptrons with two inputs, each with weight −2−2-2, and an overall bias of 333. Here's the resulting network. Note that I've moved the perceptron corresponding to the bottom right NAND gate a little, just to make it easier to draw the arrows on the diagram: One notable aspect of this network of perceptrons is that the output from the leftmost perceptron is used twice as input to the bottommost perceptron. When I defined the perceptron model I didn't say whether this kind of double-output-to-the-same-place was allowed. Actually, it doesn't much matter. If we don't want to allow this kind of thing, then it's possible to simply merge the two lines, into a single connection with a weight of -4 instead of two connections with -2 weights. (If you don't find this obvious, you should stop and prove to yourself that this is equivalent.) With that change, the network looks as follows, with all unmarked weights equal to -2, all biases equal to 3, and a single weight of -4, as marked: Up to now I've been drawing inputs like x1x1x_1 and x2x2x_2 as variables floating to the left of the network of perceptrons. In fact, it's conventional to draw an extra layer of perceptrons - the input layer - to encode the inputs: This notation for input perceptrons, in which we have an output, but no inputs, is a shorthand. It doesn't actually mean a perceptron with no inputs. To see this, suppose we did have a perceptron with no inputs. Then the weighted sum ∑jwjxj∑jwjxj\sum_j w_j x_j would always be zero, and so the perceptron would output 111 if b>0b>0b > 0, and 000 if b≤0b≤0b \leq 0. That is, the perceptron would simply output a fixed value, not the desired value (x1x1x_1, in the example above). It's better to think of the input perceptrons as not really being perceptrons at all, but rather special units which are simply defined to output the desired values, x1,x2,…x1,x2,…x_1, x_2,\ldots.The adder example demonstrates how a network of perceptrons can be used to simulate a circuit containing many NAND gates. And because NAND gates are universal for computation, it follows that perceptrons are also universal for computation.The computational universality of perceptrons is simultaneously reassuring and disappointing. It's reassuring because it tells us that networks of perceptrons can be as powerful as any other computing device. But it's also disappointing, because it makes it seem as though perceptrons are merely a new type of NAND gate. That's hardly big news!However, the situation is better than this view suggests. It turns out that we can devise learning algorithms which can automatically tune the weights and biases of a network of artificial neurons. This tuning happens in response to external stimuli, without direct intervention by a programmer. These learning algorithms enable us to use artificial neurons in a way which is radically different to conventional logic gates. Instead of explicitly laying out a circuit of NAND and other gates, our neural networks can simply learn to solve problems, sometimes problems where it would be extremely difficult to directly design a conventional circuit.Sigmoid neuronsLearning algorithms sound terrific. But how can we devise such algorithms for a neural network? Suppose we have a network of perceptrons that we'd like to use to learn to solve some problem. For example, the inputs to the network might be the raw pixel data from a scanned, handwritten image of a digit. And we'd like the network to learn weights and biases so that the output from the network correctly classifies the digit. To see how learning might work, suppose we make a small change in some weight (or bias) in the network. What we'd like is for this small change in weight to cause only a small corresponding change in the output from the network. As we'll see in a moment, this property will make learning possible. Schematically, here's what we want (obviously this network is too simple to do handwriting recognition!): If it were true that a small change in a weight (or bias) causes only a small change in output, then we could use this fact to modify the weights and biases to get our network to behave more in the manner we want. For example, suppose the network was mistakenly classifying an image as an "8" when it should be a "9". We could figure out how to make a small change in the weights and biases so the network gets a little closer to classifying the image as a "9". And then we'd repeat this, changing the weights and biases over and over to produce better and better output. The network would be learning.The problem is that this isn't what happens when our network contains perceptrons. In fact, a small change in the weights or bias of any single perceptron in the network can sometimes cause the output of that perceptron to completely flip, say from 000 to 111. That flip may then cause the behaviour of the rest of the network to completely change in some very complicated way. So while your "9" might now be classified correctly, the behaviour of the network on all the other images is likely to have completely changed in some hard-to-control way. That makes it difficult to see how to gradually modify the weights and biases so that the network gets closer to the desired behaviour. Perhaps there's some clever way of getting around this problem. But it's not immediately obvious how we can get a network of perceptrons to learn.We can overcome this problem by introducing a new type of artificial neuron called a sigmoid neuron. Sigmoid neurons are similar to perceptrons, but modified so that small changes in their weights and bias cause only a small change in their output. That's the crucial fact which will allow a network of sigmoid neurons to learn.Okay, let me describe the sigmoid neuron. We'll depict sigmoid neurons in the same way we depicted perceptrons: Just like a perceptron, the sigmoid neuron has inputs, x1,x2,…x1,x2,…x_1, x_2, \ldots. But instead of being just 000 or 111, these inputs can also take on any values between 000 and 111. So, for instance, 0.638…0.638…0.638\ldots is a valid input for a sigmoid neuron. Also just like a perceptron, the sigmoid neuron has weights for each input, w1,w2,…w1,w2,…w_1, w_2, \ldots, and an overall bias, bbb. But the output is not 000 or 111. Instead, it's σ(w⋅x+b)σ(w⋅x+b)\sigma(w \cdot x+b), where σσ\sigma is called the sigmoid function* *Incidentally, σσ\sigma is sometimes called the logistic function, and this new class of neurons called logistic neurons. It's useful to remember this terminology, since these terms are used by many people working with neural nets. However, we'll stick with the sigmoid terminology., and is defined by: σ(z)≡11+e−z.(3)(3)σ(z)≡11+e−z.\begin{eqnarray} \sigma(z) \equiv \frac{1}{1+e^{-z}}. \tag{3}\end{eqnarray} To put it all a little more explicitly, the output of a sigmoid neuron with inputs x1,x2,…x1,x2,…x_1,x_2,\ldots, weights w1,w2,…w1,w2,…w_1,w_2,\ldots, and bias bbb is 11+exp(−∑jwjxj−b).(4)(4)11+exp⁡(−∑jwjxj−b).\begin{eqnarray} \frac{1}{1+\exp(-\sum_j w_j x_j-b)}. \tag{4}\end{eqnarray}At first sight, sigmoid neurons appear very different to perceptrons. The algebraic form of the sigmoid function may seem opaque and forbidding if you're not already familiar with it. In fact, there are many similarities between perceptrons and sigmoid neurons, and the algebraic form of the sigmoid function turns out to be more of a technical detail than a true barrier to understanding.To understand the similarity to the perceptron model, suppose z≡w⋅x+bz≡w⋅x+bz \equiv w \cdot x + b is a large positive number. Then e−z≈0e−z≈0e^{-z} \approx 0 and so σ(z)≈1σ(z)≈1\sigma(z) \approx 1. In other words, when z=w⋅x+bz=w⋅x+bz = w \cdot x+b is large and positive, the output from the sigmoid neuron is approximately 111, just as it would have been for a perceptron. Suppose on the other hand that z=w⋅x+bz=w⋅x+bz = w \cdot x+b is very negative. Then e−z→∞e−z→∞e^{-z} \rightarrow \infty, and σ(z)≈0σ(z)≈0\sigma(z) \approx 0. So when z=w⋅x+bz=w⋅x+bz = w \cdot x +b is very negative, the behaviour of a sigmoid neuron also closely approximates a perceptron. It's only when w⋅x+bw⋅x+bw \cdot x+b is of modest size that there's much deviation from the perceptron model.What about the algebraic form of σσ\sigma? How can we understand that? In fact, the exact form of σσ\sigma isn't so important - what really matters is the shape of the function when plotted. Here's the shape: -4-3-2-1012340.00.20.40.60.81.0zsigmoid function function s(x) {return 1/(1+Math.exp(-x));} var m = [40, 120, 50, 120]; var height = 290 - m[0] - m[2]; var width = 600 - m[1] - m[3]; var xmin = -5; var xmax = 5; var sample = 400; var x1 = d3.scale.linear().domain([0, sample]).range([xmin, xmax]); var data = d3.range(sample).map(function(d){ return { x: x1(d), y: s(x1(d))}; }); var x = d3.scale.linear().domain([xmin, xmax]).range([0, width]); var y = d3.scale.linear() .domain([0, 1]) .range([height, 0]); var line = d3.svg.line() .x(function(d) { return x(d.x); }) .y(function(d) { return y(d.y); }) var graph = d3.select("#sigmoid_graph") .append("svg") .attr("width", width + m[1] + m[3]) .attr("height", height + m[0] + m[2]) .append("g") .attr("transform", "translate(" + m[3] + "," + m[0] + ")"); var xAxis = d3.svg.axis() .scale(x) .tickValues(d3.range(-4, 5, 1)) .orient("bottom") graph.append("g") .attr("class", "x axis") .attr("transform", "translate(0, " + height + ")") .call(xAxis); var yAxis = d3.svg.axis() .scale(y) .tickValues(d3.range(0, 1.01, 0.2)) .orient("left") .ticks(5) graph.append("g") .attr("class", "y axis") .call(yAxis); graph.append("path").attr("d", line(data)); graph.append("text") .attr("class", "x label") .attr("text-anchor", "end") .attr("x", width/2) .attr("y", height+35) .text("z"); graph.append("text") .attr("x", (width / 2)) .attr("y", -10) .attr("text-anchor", "middle") .style("font-size", "16px") .text("sigmoid function"); This shape is a smoothed out version of a step function: -4-3-2-1012340.00.20.40.60.81.0zstep function function s(x) {return x < 0 ? 0 : 1;} var m = [40, 120, 50, 120]; var height = 290 - m[0] - m[2]; var width = 600 - m[1] - m[3]; var xmin = -5; var xmax = 5; var sample = 400; var x1 = d3.scale.linear().domain([0, sample]).range([xmin, xmax]); var data = d3.range(sample).map(function(d){ return { x: x1(d), y: s(x1(d))}; }); var x = d3.scale.linear().domain([xmin, xmax]).range([0, width]); var y = d3.scale.linear() .domain([0,1]) .range([height, 0]); var line = d3.svg.line() .x(function(d) { return x(d.x); }) .y(function(d) { return y(d.y); }) var graph = d3.select("#step_graph") .append("svg") .attr("width", width + m[1] + m[3]) .attr("height", height + m[0] + m[2]) .append("g") .attr("transform", "translate(" + m[3] + "," + m[0] + ")"); var xAxis = d3.svg.axis() .scale(x) .tickValues(d3.range(-4, 5, 1)) .orient("bottom") graph.append("g") .attr("class", "x axis") .attr("transform", "translate(0, " + height + ")") .call(xAxis); var yAxis = d3.svg.axis() .scale(y) .tickValues(d3.range(0, 1.01, 0.2)) .orient("left") .ticks(5) graph.append("g") .attr("class", "y axis") .call(yAxis); graph.append("path").attr("d", line(data)); graph.append("text") .attr("class", "x label") .attr("text-anchor", "end") .attr("x", width/2) .attr("y", height+35) .text("z"); graph.append("text") .attr("x", (width / 2)) .attr("y", -10) .attr("text-anchor", "middle") .style("font-size", "16px") .text("step function"); If σσ\sigma had in fact been a step function, then the sigmoid neuron would be a perceptron, since the output would be 111 or 000 depending on whether w⋅x+bw⋅x+bw\cdot x+b was positive or negative* *Actually, when w⋅x+b=0w⋅x+b=0w \cdot x +b = 0 the perceptron outputs 000, while the step function outputs 111. So, strictly speaking, we'd need to modify the step function at that one point. But you get the idea.. By using the actual σσ\sigma function we get, as already implied above, a smoothed out perceptron. Indeed, it's the smoothness of the σσ\sigma function that is the crucial fact, not its detailed form. The smoothness of σσ\sigma means that small changes ΔwjΔwj\Delta w_j in the weights and ΔbΔb\Delta b in the bias will produce a small change ΔoutputΔoutput\Delta \mbox{output} in the output from the neuron. In fact, calculus tells us that ΔoutputΔoutput\Delta \mbox{output} is well approximated by Δoutput≈∑j∂output∂wjΔwj+∂output∂bΔb,(5)(5)Δoutput≈∑j∂output∂wjΔwj+∂output∂bΔb,\begin{eqnarray} \Delta \mbox{output} \approx \sum_j \frac{\partial \, \mbox{output}}{\partial w_j} \Delta w_j + \frac{\partial \, \mbox{output}}{\partial b} \Delta b, \tag{5}\end{eqnarray} where the sum is over all the weights, wjwjw_j, and ∂output/∂wj∂output/∂wj\partial \, \mbox{output} / \partial w_j and ∂output/∂b∂output/∂b\partial \, \mbox{output} /\partial b denote partial derivatives of the outputoutput\mbox{output} with respect to wjwjw_j and bbb, respectively. Don't panic if you're not comfortable with partial derivatives! While the expression above looks complicated, with all the partial derivatives, it's actually saying something very simple (and which is very good news): ΔoutputΔoutput\Delta \mbox{output} is a linear function of the changes ΔwjΔwj\Delta w_j and ΔbΔb\Delta b in the weights and bias. This linearity makes it easy to choose small changes in the weights and biases to achieve any desired small change in the output. So while sigmoid neurons have much of the same qualitative behaviour as perceptrons, they make it much easier to figure out how changing the weights and biases will change the output.If it's the shape of σσ\sigma which really matters, and not its exact form, then why use the particular form used for σσ\sigma in Equation (3)σ(z)≡11+e−zσ(z)≡11+e−z\begin{eqnarray} \sigma(z) \equiv \frac{1}{1+e^{-z}} \nonumber\end{eqnarray}$('#margin_387419264610_reveal').click(function() {$('#margin_387419264610').toggle('slow', function() {});});? In fact, later in the book we will occasionally consider neurons where the output is f(w⋅x+b)f(w⋅x+b)f(w \cdot x + b) for some other activation function f(⋅)f(⋅)f(\cdot). The main thing that changes when we use a different activation function is that the particular values for the partial derivatives in Equation (5)Δoutput≈∑j∂output∂wjΔwj+∂output∂bΔbΔoutput≈∑j∂output∂wjΔwj+∂output∂bΔb\begin{eqnarray} \Delta \mbox{output} \approx \sum_j \frac{\partial \, \mbox{output}}{\partial w_j} \Delta w_j + \frac{\partial \, \mbox{output}}{\partial b} \Delta b \nonumber\end{eqnarray}$('#margin_727997094331_reveal').click(function() {$('#margin_727997094331').toggle('slow', function() {});}); change. It turns out that when we compute those partial derivatives later, using σσ\sigma will simplify the algebra, simply because exponentials have lovely properties when differentiated. In any case, σσ\sigma is commonly-used in work on neural nets, and is the activation function we'll use most often in this book.How should we interpret the output from a sigmoid neuron? Obviously, one big difference between perceptrons and sigmoid neurons is that sigmoid neurons don't just output 000 or 111. They can have as output any real number between 000 and 111, so values such as 0.173…0.173…0.173\ldots and 0.689…0.689…0.689\ldots are legitimate outputs. This can be useful, for example, if we want to use the output value to represent the average intensity of the pixels in an image input to a neural network. But sometimes it can be a nuisance. Suppose we want the output from the network to indicate either "the input image is a 9" or "the input image is not a 9". Obviously, it'd be easiest to do this if the output was a 000 or a 111, as in a perceptron. But in practice we can set up a convention to deal with this, for example, by deciding to interpret any output of at least 0.50.50.5 as indicating a "9", and any output less than 0.50.50.5 as indicating "not a 9". I'll always explicitly state when we're using such a convention, so it shouldn't cause any confusion. Exercises Sigmoid neurons simulating perceptrons, part I \mbox{} Suppose we take all the weights and biases in a network of perceptrons, and multiply them by a positive constant, c>0c>0c > 0. Show that the behaviour of the network doesn't change.Sigmoid neurons simulating perceptrons, part II \mbox{} Suppose we have the same setup as the last problem - a network of perceptrons. Suppose also that the overall input to the network of perceptrons has been chosen. We won't need the actual input value, we just need the input to have been fixed. Suppose the weights and biases are such that w⋅x+b≠0w⋅x+b≠0w \cdot x + b \neq 0 for the input xxx to any particular perceptron in the network. Now replace all the perceptrons in the network by sigmoid neurons, and multiply the weights and biases by a positive constant c>0c>0c > 0. Show that in the limit as c→∞c→∞c \rightarrow \infty the behaviour of this network of sigmoid neurons is exactly the same as the network of perceptrons. How can this fail when w⋅x+b=0w⋅x+b=0w \cdot x + b = 0 for one of the perceptrons? The architecture of neural networksIn the next section I'll introduce a neural network that can do a pretty good job classifying handwritten digits. In preparation for that, it helps to explain some terminology that lets us name different parts of a network. Suppose we have the network: As mentioned earlier, the leftmost layer in this network is called the input layer, and the neurons within the layer are called input neurons. The rightmost or output layer contains the output neurons, or, as in this case, a single output neuron. The middle layer is called a hidden layer, since the neurons in this layer are neither inputs nor outputs. The term "hidden" perhaps sounds a little mysterious - the first time I heard the term I thought it must have some deep philosophical or mathematical significance - but it really means nothing more than "not an input or an output". The network above has just a single hidden layer, but some networks have multiple hidden layers. For example, the following four-layer network has two hidden layers: Somewhat confusingly, and for historical reasons, such multiple layer networks are sometimes called multilayer perceptrons or MLPs, despite being made up of sigmoid neurons, not perceptrons. I'm not going to use the MLP terminology in this book, since I think it's confusing, but wanted to warn you of its existence.The design of the input and output layers in a network is often straightforward. For example, suppose we're trying to determine whether a handwritten image depicts a "9" or not. A natural way to design the network is to encode the intensities of the image pixels into the input neurons. If the image is a 646464 by 646464 greyscale image, then we'd have 4,096=64×644,096=64×644,096 = 64 \times 64 input neurons, with the intensities scaled appropriately between 000 and 111. The output layer will contain just a single neuron, with output values of less than 0.50.50.5 indicating "input image is not a 9", and values greater than 0.50.50.5 indicating "input image is a 9 ". While the design of the input and output layers of a neural network is often straightforward, there can be quite an art to the design of the hidden layers. In particular, it's not possible to sum up the design process for the hidden layers with a few simple rules of thumb. Instead, neural networks researchers have developed many design heuristics for the hidden layers, which help people get the behaviour they want out of their nets. For example, such heuristics can be used to help determine how to trade off the number of hidden layers against the time required to train the network. We'll meet several such design heuristics later in this book. Up to now, we've been discussing neural networks where the output from one layer is used as input to the next layer. Such networks are called feedforward neural networks. This means there are no loops in the network - information is always fed forward, never fed back. If we did have loops, we'd end up with situations where the input to the σσ\sigma function depended on the output. That'd be hard to make sense of, and so we don't allow such loops.However, there are other models of artificial neural networks in which feedback loops are possible. These models are called recurrent neural networks. The idea in these models is to have neurons which fire for some limited duration of time, before becoming quiescent. That firing can stimulate other neurons, which may fire a little while later, also for a limited duration. That causes still more neurons to fire, and so over time we get a cascade of neurons firing. Loops don't cause problems in such a model, since a neuron's output only affects its input at some later time, not instantaneously.Recurrent neural nets have been less influential than feedforward networks, in part because the learning algorithms for recurrent nets are (at least to date) less powerful. But recurrent networks are still extremely interesting. They're much closer in spirit to how our brains work than feedforward networks. And it's possible that recurrent networks can solve important problems which can only be solved with great difficulty by feedforward networks. However, to limit our scope, in this book we're going to concentrate on the more widely-used feedforward networks.A simple network to classify handwritten digitsHaving defined neural networks, let's return to handwriting recognition. We can split the problem of recognizing handwritten digits into two sub-problems. First, we'd like a way of breaking an image containing many digits into a sequence of separate images, each containing a single digit. For example, we'd like to break the imageinto six separate images, We humans solve this segmentation problem with ease, but it's challenging for a computer program to correctly break up the image. Once the image has been segmented, the program then needs to classify each individual digit. So, for instance, we'd like our program to recognize that the first digit above,is a 5.We'll focus on writing a program to solve the second problem, that is, classifying individual digits. We do this because it turns out that the segmentation problem is not so difficult to solve, once you have a good way of classifying individual digits. There are many approaches to solving the segmentation problem. One approach is to trial many different ways of segmenting the image, using the individual digit classifier to score each trial segmentation. A trial segmentation gets a high score if the individual digit classifier is confident of its classification in all segments, and a low score if the classifier is having a lot of trouble in one or more segments. The idea is that if the classifier is having trouble somewhere, then it's probably having trouble because the segmentation has been chosen incorrectly. This idea and other variations can be used to solve the segmentation problem quite well. So instead of worrying about segmentation we'll concentrate on developing a neural network which can solve the more interesting and difficult problem, namely, recognizing individual handwritten digits.To recognize individual digits we will use a three-layer neural network: The input layer of the network contains neurons encoding the values of the input pixels. As discussed in the next section, our training data for the network will consist of many 282828 by 282828 pixel images of scanned handwritten digits, and so the input layer contains 784=28×28784=28×28784 = 28 \times 28 neurons. For simplicity I've omitted most of the 784784784 input neurons in the diagram above. The input pixels are greyscale, with a value of 0.00.00.0 representing white, a value of 1.01.01.0 representing black, and in between values representing gradually darkening shades of grey.The second layer of the network is a hidden layer. We denote the number of neurons in this hidden layer by nnn, and we'll experiment with different values for nnn. The example shown illustrates a small hidden layer, containing just n=15n=15n = 15 neurons.The output layer of the network contains 10 neurons. If the first neuron fires, i.e., has an output ≈1≈1\approx 1, then that will indicate that the network thinks the digit is a 000. If the second neuron fires then that will indicate that the network thinks the digit is a 111. And so on. A little more precisely, we number the output neurons from 000 through 999, and figure out which neuron has the highest activation value. If that neuron is, say, neuron number 666, then our network will guess that the input digit was a 666. And so on for the other output neurons.You might wonder why we use 101010 output neurons. After all, the goal of the network is to tell us which digit (0,1,2,…,90,1,2,…,90, 1, 2, \ldots, 9) corresponds to the input image. A seemingly natural way of doing that is to use just 444 output neurons, treating each neuron as taking on a binary value, depending on whether the neuron's output is closer to 000 or to 111. Four neurons are enough to encode the answer, since 24=1624=162^4 = 16 is more than the 10 possible values for the input digit. Why should our network use 101010 neurons instead? Isn't that inefficient? The ultimate justification is empirical: we can try out both network designs, and it turns out that, for this particular problem, the network with 101010 output neurons learns to recognize digits better than the network with 444 output neurons. But that leaves us wondering why using 101010 output neurons works better. Is there some heuristic that would tell us in advance that we should use the 101010-output encoding instead of the 444-output encoding?To understand why we do this, it helps to think about what the neural network is doing from first principles. Consider first the case where we use 101010 output neurons. Let's concentrate on the first output neuron, the one that's trying to decide whether or not the digit is a 000. It does this by weighing up evidence from the hidden layer of neurons. What are those hidden neurons doing? Well, just suppose for the sake of argument that the first neuron in the hidden layer detects whether or not an image like the following is present:It can do this by heavily weighting input pixels which overlap with the image, and only lightly weighting the other inputs. In a similar way, let's suppose for the sake of argument that the second, third, and fourth neurons in the hidden layer detect whether or not the following images are present:As you may have guessed, these four images together make up the 000 image that we saw in the line of digits shown earlier:So if all four of these hidden neurons are firing then we can conclude that the digit is a 000. Of course, that's not the only sort of evidence we can use to conclude that the image was a 000 - we could legitimately get a 000 in many other ways (say, through translations of the above images, or slight distortions). But it seems safe to say that at least in this case we'd conclude that the input was a 000.Supposing the neural network functions in this way, we can give a plausible explanation for why it's better to have 101010 outputs from the network, rather than 444. If we had 444 outputs, then the first output neuron would be trying to decide what the most significant bit of the digit was. And there's no easy way to relate that most significant bit to simple shapes like those shown above. It's hard to imagine that there's any good historical reason the component shapes of the digit will be closely related to (say) the most significant bit in the output.Now, with all that said, this is all just a heuristic. Nothing says that the three-layer neural network has to operate in the way I described, with the hidden neurons detecting simple component shapes. Maybe a clever learning algorithm will find some assignment of weights that lets us use only 444 output neurons. But as a heuristic the way of thinking I've described works pretty well, and can save you a lot of time in designing good neural network architectures.Exercise There is a way of determining the bitwise representation of a digit by adding an extra layer to the three-layer network above. The extra layer converts the output from the previous layer into a binary representation, as illustrated in the figure below. Find a set of weights and biases for the new output layer. Assume that the first 333 layers of neurons are such that the correct output in the third layer (i.e., the old output layer) has activation at least 0.990.990.99, and incorrect outputs have activation less than 0.010.010.01. Learning with gradient descentNow that we have a design for our neural network, how can it learn to recognize digits? The first thing we'll need is a data set to learn from - a so-called training data set. We'll use the MNIST data set, which contains tens of thousands of scanned images of handwritten digits, together with their correct classifications. MNIST's name comes from the fact that it is a modified subset of two data sets collected by NIST, the United States' National Institute of Standards and Technology. Here's a few images from MNIST: As you can see, these digits are, in fact, the same as those shown at the beginning of this chapter as a challenge to recognize. Of course, when testing our network we'll ask it to recognize images which aren't in the training set!The MNIST data comes in two parts. The first part contains 60,000 images to be used as training data. These images are scanned handwriting samples from 250 people, half of whom were US Census Bureau employees, and half of whom were high school students. The images are greyscale and 28 by 28 pixels in size. The second part of the MNIST data set is 10,000 images to be used as test data. Again, these are 28 by 28 greyscale images. We'll use the test data to evaluate how well our neural network has learned to recognize digits. To make this a good test of performance, the test data was taken from a different set of 250 people than the original training data (albeit still a group split between Census Bureau employees and high school students). This helps give us confidence that our system can recognize digits from people whose writing it didn't see during training.We'll use the notation xxx to denote a training input. It'll be convenient to regard each training input xxx as a 28×28=78428×28=78428 \times 28 = 784-dimensional vector. Each entry in the vector represents the grey value for a single pixel in the image. We'll denote the corresponding desired output by y=y(x)y=y(x)y = y(x), where yyy is a 101010-dimensional vector. For example, if a particular training image, xxx, depicts a 666, then y(x)=(0,0,0,0,0,0,1,0,0,0)Ty(x)=(0,0,0,0,0,0,1,0,0,0)Ty(x) = (0, 0, 0, 0, 0, 0, 1, 0, 0, 0)^T is the desired output from the network. Note that TTT here is the transpose operation, turning a row vector into an ordinary (column) vector.What we'd like is an algorithm which lets us find weights and biases so that the output from the network approximates y(x)y(x)y(x) for all training inputs xxx. To quantify how well we're achieving this goal we define a cost function* *Sometimes referred to as a loss or objective function. We use the term cost function throughout this book, but you should note the other terminology, since it's often used in research papers and other discussions of neural networks. : C(w,b)≡12n∑x∥y(x)−a∥2.(6)(6)C(w,b)≡12n∑x‖y(x)−a‖2.\begin{eqnarray} C(w,b) \equiv \frac{1}{2n} \sum_x \| y(x) - a\|^2. \tag{6}\end{eqnarray} Here, www denotes the collection of all weights in the network, bbb all the biases, nnn is the total number of training inputs, aaa is the vector of outputs from the network when xxx is input, and the sum is over all training inputs, xxx. Of course, the output aaa depends on xxx, www and bbb, but to keep the notation simple I haven't explicitly indicated this dependence. The notation ∥v∥‖v‖\| v \| just denotes the usual length function for a vector vvv. We'll call CCC the quadratic cost function; it's also sometimes known as the mean squared error or just MSE. Inspecting the form of the quadratic cost function, we see that C(w,b)C(w,b)C(w,b) is non-negative, since every term in the sum is non-negative. Furthermore, the cost C(w,b)C(w,b)C(w,b) becomes small, i.e., C(w,b)≈0C(w,b)≈0C(w,b) \approx 0, precisely when y(x)y(x)y(x) is approximately equal to the output, aaa, for all training inputs, xxx. So our training algorithm has done a good job if it can find weights and biases so that C(w,b)≈0C(w,b)≈0C(w,b) \approx 0. By contrast, it's not doing so well when C(w,b)C(w,b)C(w,b) is large - that would mean that y(x)y(x)y(x) is not close to the output aaa for a large number of inputs. So the aim of our training algorithm will be to minimize the cost C(w,b)C(w,b)C(w,b) as a function of the weights and biases. In other words, we want to find a set of weights and biases which make the cost as small as possible. We'll do that using an algorithm known as gradient descent. Why introduce the quadratic cost? After all, aren't we primarily interested in the number of images correctly classified by the network? Why not try to maximize that number directly, rather than minimizing a proxy measure like the quadratic cost? The problem with that is that the number of images correctly classified is not a smooth function of the weights and biases in the network. For the most part, making small changes to the weights and biases won't cause any change at all in the number of training images classified correctly. That makes it difficult to figure out how to change the weights and biases to get improved performance. If we instead use a smooth cost function like the quadratic cost it turns out to be easy to figure out how to make small changes in the weights and biases so as to get an improvement in the cost. That's why we focus first on minimizing the quadratic cost, and only after that will we examine the classification accuracy.Even given that we want to use a smooth cost function, you may still wonder why we choose the quadratic function used in Equation (6)C(w,b)≡12n∑x∥y(x)−a∥2C(w,b)≡12n∑x‖y(x)−a‖2\begin{eqnarray} C(w,b) \equiv \frac{1}{2n} \sum_x \| y(x) - a\|^2 \nonumber\end{eqnarray}$('#margin_501822820305_reveal').click(function() {$('#margin_501822820305').toggle('slow', function() {});});. Isn't this a rather ad hoc choice? Perhaps if we chose a different cost function we'd get a totally different set of minimizing weights and biases? This is a valid concern, and later we'll revisit the cost function, and make some modifications. However, the quadratic cost function of Equation (6)C(w,b)≡12n∑x∥y(x)−a∥2C(w,b)≡12n∑x‖y(x)−a‖2\begin{eqnarray} C(w,b) \equiv \frac{1}{2n} \sum_x \| y(x) - a\|^2 \nonumber\end{eqnarray}$('#margin_555483302348_reveal').click(function() {$('#margin_555483302348').toggle('slow', function() {});}); works perfectly well for understanding the basics of learning in neural networks, so we'll stick with it for now.Recapping, our goal in training a neural network is to find weights and biases which minimize the quadratic cost function C(w,b)C(w,b)C(w, b). This is a well-posed problem, but it's got a lot of distracting structure as currently posed - the interpretation of www and bbb as weights and biases, the σσ\sigma function lurking in the background, the choice of network architecture, MNIST, and so on. It turns out that we can understand a tremendous amount by ignoring most of that structure, and just concentrating on the minimization aspect. So for now we're going to forget all about the specific form of the cost function, the connection to neural networks, and so on. Instead, we're going to imagine that we've simply been given a function of many variables and we want to minimize that function. We're going to develop a technique called gradient descent which can be used to solve such minimization problems. Then we'll come back to the specific function we want to minimize for neural networks.Okay, let's suppose we're trying to minimize some function, C(v)C(v)C(v). This could be any real-valued function of many variables, v=v1,v2,…v=v1,v2,…v = v_1, v_2, \ldots. Note that I've replaced the www and bbb notation by vvv to emphasize that this could be any function - we're not specifically thinking in the neural networks context any more. To minimize C(v)C(v)C(v) it helps to imagine CCC as a function of just two variables, which we'll call v1v1v_1 and v2v2v_2:What we'd like is to find where CCC achieves its global minimum. Now, of course, for the function plotted above, we can eyeball the graph and find the minimum. In that sense, I've perhaps shown slightly too simple a function! A general function, CCC, may be a complicated function of many variables, and it won't usually be possible to just eyeball the graph to find the minimum.One way of attacking the problem is to use calculus to try to find the minimum analytically. We could compute derivatives and then try using them to find places where CCC is an extremum. With some luck that might work when CCC is a function of just one or a few variables. But it'll turn into a nightmare when we have many more variables. And for neural networks we'll often want far more variables - the biggest neural networks have cost functions which depend on billions of weights and biases in an extremely complicated way. Using calculus to minimize that just won't work!(After asserting that we'll gain insight by imagining CCC as a function of just two variables, I've turned around twice in two paragraphs and said, "hey, but what if it's a function of many more than two variables?" Sorry about that. Please believe me when I say that it really does help to imagine CCC as a function of two variables. It just happens that sometimes that picture breaks down, and the last two paragraphs were dealing with such breakdowns. Good thinking about mathematics often involves juggling multiple intuitive pictures, learning when it's appropriate to use each picture, and when it's not.)Okay, so calculus doesn't work. Fortunately, there is a beautiful analogy which suggests an algorithm which works pretty well. We start by thinking of our function as a kind of a valley. If you squint just a little at the plot above, that shouldn't be too hard. And we imagine a ball rolling down the slope of the valley. Our everyday experience tells us that the ball will eventually roll to the bottom of the valley. Perhaps we can use this idea as a way to find a minimum for the function? We'd randomly choose a starting point for an (imaginary) ball, and then simulate the motion of the ball as it rolled down to the bottom of the valley. We could do this simulation simply by computing derivatives (and perhaps some second derivatives) of CCC - those derivatives would tell us everything we need to know about the local "shape" of the valley, and therefore how our ball should roll.Based on what I've just written, you might suppose that we'll be trying to write down Newton's equations of motion for the ball, considering the effects of friction and gravity, and so on. Actually, we're not going to take the ball-rolling analogy quite that seriously - we're devising an algorithm to minimize CCC, not developing an accurate simulation of the laws of physics! The ball's-eye view is meant to stimulate our imagination, not constrain our thinking. So rather than get into all the messy details of physics, let's simply ask ourselves: if we were declared God for a day, and could make up our own laws of physics, dictating to the ball how it should roll, what law or laws of motion could we pick that would make it so the ball always rolled to the bottom of the valley?To make this question more precise, let's think about what happens when we move the ball a small amount Δv1Δv1\Delta v_1 in the v1v1v_1 direction, and a small amount Δv2Δv2\Delta v_2 in the v2v2v_2 direction. Calculus tells us that CCC changes as follows: ΔC≈∂C∂v1Δv1+∂C∂v2Δv2.(7)(7)ΔC≈∂C∂v1Δv1+∂C∂v2Δv2.\begin{eqnarray} \Delta C \approx \frac{\partial C}{\partial v_1} \Delta v_1 + \frac{\partial C}{\partial v_2} \Delta v_2. \tag{7}\end{eqnarray} We're going to find a way of choosing Δv1Δv1\Delta v_1 and Δv2Δv2\Delta v_2 so as to make ΔCΔC\Delta C negative; i.e., we'll choose them so the ball is rolling down into the valley. To figure out how to make such a choice it helps to define ΔvΔv\Delta v to be the vector of changes in vvv, Δv≡(Δv1,Δv2)TΔv≡(Δv1,Δv2)T\Delta v \equiv (\Delta v_1, \Delta v_2)^T, where TTT is again the transpose operation, turning row vectors into column vectors. We'll also define the gradient of CCC to be the vector of partial derivatives, (∂C∂v1,∂C∂v2)T(∂C∂v1,∂C∂v2)T\left(\frac{\partial C}{\partial v_1}, \frac{\partial C}{\partial v_2}\right)^T. We denote the gradient vector by ∇C∇C\nabla C, i.e.: ∇C≡(∂C∂v1,∂C∂v2)T.(8)(8)∇C≡(∂C∂v1,∂C∂v2)T.\begin{eqnarray} \nabla C \equiv \left( \frac{\partial C}{\partial v_1}, \frac{\partial C}{\partial v_2} \right)^T. \tag{8}\end{eqnarray} In a moment we'll rewrite the change ΔCΔC\Delta C in terms of ΔvΔv\Delta v and the gradient, ∇C∇C\nabla C. Before getting to that, though, I want to clarify something that sometimes gets people hung up on the gradient. When meeting the ∇C∇C\nabla C notation for the first time, people sometimes wonder how they should think about the ∇∇\nabla symbol. What, exactly, does ∇∇\nabla mean? In fact, it's perfectly fine to think of ∇C∇C\nabla C as a single mathematical object - the vector defined above - which happens to be written using two symbols. In this point of view, ∇∇\nabla is just a piece of notational flag-waving, telling you "hey, ∇C∇C\nabla C is a gradient vector". There are more advanced points of view where ∇∇\nabla can be viewed as an independent mathematical entity in its own right (for example, as a differential operator), but we won't need such points of view.With these definitions, the expression (7)ΔC≈∂C∂v1Δv1+∂C∂v2Δv2ΔC≈∂C∂v1Δv1+∂C∂v2Δv2\begin{eqnarray} \Delta C \approx \frac{\partial C}{\partial v_1} \Delta v_1 + \frac{\partial C}{\partial v_2} \Delta v_2 \nonumber\end{eqnarray}$('#margin_512380394946_reveal').click(function() {$('#margin_512380394946').toggle('slow', function() {});}); for ΔCΔC\Delta C can be rewritten as ΔC≈∇C⋅Δv.(9)(9)ΔC≈∇C⋅Δv.\begin{eqnarray} \Delta C \approx \nabla C \cdot \Delta v. \tag{9}\end{eqnarray} This equation helps explain why ∇C∇C\nabla C is called the gradient vector: ∇C∇C\nabla C relates changes in vvv to changes in CCC, just as we'd expect something called a gradient to do. But what's really exciting about the equation is that it lets us see how to choose ΔvΔv\Delta v so as to make ΔCΔC\Delta C negative. In particular, suppose we choose Δv=−η∇C,(10)(10)Δv=−η∇C,\begin{eqnarray} \Delta v = -\eta \nabla C, \tag{10}\end{eqnarray} where ηη\eta is a small, positive parameter (known as the learning rate). Then Equation (9)ΔC≈∇C⋅ΔvΔC≈∇C⋅Δv\begin{eqnarray} \Delta C \approx \nabla C \cdot \Delta v \nonumber\end{eqnarray}$('#margin_31741254841_reveal').click(function() {$('#margin_31741254841').toggle('slow', function() {});}); tells us that ΔC≈−η∇C⋅∇C=−η∥∇C∥2ΔC≈−η∇C⋅∇C=−η‖∇C‖2\Delta C \approx -\eta \nabla C \cdot \nabla C = -\eta \|\nabla C\|^2. Because ∥∇C∥2≥0‖∇C‖2≥0\| \nabla C \|^2 \geq 0, this guarantees that ΔC≤0ΔC≤0\Delta C \leq 0, i.e., CCC will always decrease, never increase, if we change vvv according to the prescription in (10)Δv=−η∇CΔv=−η∇C\begin{eqnarray} \Delta v = -\eta \nabla C \nonumber\end{eqnarray}$('#margin_48762573303_reveal').click(function() {$('#margin_48762573303').toggle('slow', function() {});});. (Within, of course, the limits of the approximation in Equation (9)ΔC≈∇C⋅ΔvΔC≈∇C⋅Δv\begin{eqnarray} \Delta C \approx \nabla C \cdot \Delta v \nonumber\end{eqnarray}$('#margin_919658643545_reveal').click(function() {$('#margin_919658643545').toggle('slow', function() {});});). This is exactly the property we wanted! And so we'll take Equation (10)Δv=−η∇CΔv=−η∇C\begin{eqnarray} \Delta v = -\eta \nabla C \nonumber\end{eqnarray}$('#margin_287729255111_reveal').click(function() {$('#margin_287729255111').toggle('slow', function() {});}); to define the "law of motion" for the ball in our gradient descent algorithm. That is, we'll use Equation (10)Δv=−η∇CΔv=−η∇C\begin{eqnarray} \Delta v = -\eta \nabla C \nonumber\end{eqnarray}$('#margin_718723868298_reveal').click(function() {$('#margin_718723868298').toggle('slow', function() {});}); to compute a value for ΔvΔv\Delta v, then move the ball's position vvv by that amount: v→v′=v−η∇C.(11)(11)v→v′=v−η∇C.\begin{eqnarray} v \rightarrow v' = v -\eta \nabla C. \tag{11}\end{eqnarray} Then we'll use this update rule again, to make another move. If we keep doing this, over and over, we'll keep decreasing CCC until - we hope - we reach a global minimum.Summing up, the way the gradient descent algorithm works is to repeatedly compute the gradient ∇C∇C\nabla C, and then to move in the opposite direction, "falling down" the slope of the valley. We can visualize it like this:Notice that with this rule gradient descent doesn't reproduce real physical motion. In real life a ball has momentum, and that momentum may allow it to roll across the slope, or even (momentarily) roll uphill. It's only after the effects of friction set in that the ball is guaranteed to roll down into the valley. By contrast, our rule for choosing ΔvΔv\Delta v just says "go down, right now". That's still a pretty good rule for finding the minimum!To make gradient descent work correctly, we need to choose the learning rate ηη\eta to be small enough that Equation (9)ΔC≈∇C⋅ΔvΔC≈∇C⋅Δv\begin{eqnarray} \Delta C \approx \nabla C \cdot \Delta v \nonumber\end{eqnarray}$('#margin_560455937071_reveal').click(function() {$('#margin_560455937071').toggle('slow', function() {});}); is a good approximation. If we don't, we might end up with ΔC>0ΔC>0\Delta C > 0, which obviously would not be good! At the same time, we don't want ηη\eta to be too small, since that will make the changes ΔvΔv\Delta v tiny, and thus the gradient descent algorithm will work very slowly. In practical implementations, ηη\eta is often varied so that Equation (9)ΔC≈∇C⋅ΔvΔC≈∇C⋅Δv\begin{eqnarray} \Delta C \approx \nabla C \cdot \Delta v \nonumber\end{eqnarray}$('#margin_157848846275_reveal').click(function() {$('#margin_157848846275').toggle('slow', function() {});}); remains a good approximation, but the algorithm isn't too slow. We'll see later how this works. I've explained gradient descent when CCC is a function of just two variables. But, in fact, everything works just as well even when CCC is a function of many more variables. Suppose in particular that CCC is a function of mmm variables, v1,…,vmv1,…,vmv_1,\ldots,v_m. Then the change ΔCΔC\Delta C in CCC produced by a small change Δv=(Δv1,…,Δvm)TΔv=(Δv1,…,Δvm)T\Delta v = (\Delta v_1, \ldots, \Delta v_m)^T is ΔC≈∇C⋅Δv,(12)(12)ΔC≈∇C⋅Δv,\begin{eqnarray} \Delta C \approx \nabla C \cdot \Delta v, \tag{12}\end{eqnarray} where the gradient ∇C∇C\nabla C is the vector ∇C≡(∂C∂v1,…,∂C∂vm)T.(13)(13)∇C≡(∂C∂v1,…,∂C∂vm)T.\begin{eqnarray} \nabla C \equiv \left(\frac{\partial C}{\partial v_1}, \ldots, \frac{\partial C}{\partial v_m}\right)^T. \tag{13}\end{eqnarray} Just as for the two variable case, we can choose Δv=−η∇C,(14)(14)Δv=−η∇C,\begin{eqnarray} \Delta v = -\eta \nabla C, \tag{14}\end{eqnarray} and we're guaranteed that our (approximate) expression (12)ΔC≈∇C⋅ΔvΔC≈∇C⋅Δv\begin{eqnarray} \Delta C \approx \nabla C \cdot \Delta v \nonumber\end{eqnarray}$('#margin_869505431896_reveal').click(function() {$('#margin_869505431896').toggle('slow', function() {});}); for ΔCΔC\Delta C will be negative. This gives us a way of following the gradient to a minimum, even when CCC is a function of many variables, by repeatedly applying the update rule v→v′=v−η∇C.(15)(15)v→v′=v−η∇C.\begin{eqnarray} v \rightarrow v' = v-\eta \nabla C. \tag{15}\end{eqnarray} You can think of this update rule as defining the gradient descent algorithm. It gives us a way of repeatedly changing the position vvv in order to find a minimum of the function CCC. The rule doesn't always work - several things can go wrong and prevent gradient descent from finding the global minimum of CCC, a point we'll return to explore in later chapters. But, in practice gradient descent often works extremely well, and in neural networks we'll find that it's a powerful way of minimizing the cost function, and so helping the net learn.Indeed, there's even a sense in which gradient descent is the optimal strategy for searching for a minimum. Let's suppose that we're trying to make a move ΔvΔv\Delta v in position so as to decrease CCC as much as possible. This is equivalent to minimizing ΔC≈∇C⋅ΔvΔC≈∇C⋅Δv\Delta C \approx \nabla C \cdot \Delta v. We'll constrain the size of the move so that ∥Δv∥=ϵ‖Δv‖=ϵ\| \Delta v \| = \epsilon for some small fixed ϵ>0ϵ>0\epsilon > 0. In other words, we want a move that is a small step of a fixed size, and we're trying to find the movement direction which decreases CCC as much as possible. It can be proved that the choice of ΔvΔv\Delta v which minimizes ∇C⋅Δv∇C⋅Δv\nabla C \cdot \Delta v is Δv=−η∇CΔv=−η∇C\Delta v = - \eta \nabla C, where η=ϵ/∥∇C∥η=ϵ/‖∇C‖\eta = \epsilon / \|\nabla C\| is determined by the size constraint ∥Δv∥=ϵ‖Δv‖=ϵ\|\Delta v\| = \epsilon. So gradient descent can be viewed as a way of taking small steps in the direction which does the most to immediately decrease CCC.Exercises Prove the assertion of the last paragraph. Hint: If you're not already familiar with the Cauchy-Schwarz inequality, you may find it helpful to familiarize yourself with it. I explained gradient descent when CCC is a function of two variables, and when it's a function of more than two variables. What happens when CCC is a function of just one variable? Can you provide a geometric interpretation of what gradient descent is doing in the one-dimensional case? People have investigated many variations of gradient descent, including variations that more closely mimic a real physical ball. These ball-mimicking variations have some advantages, but also have a major disadvantage: it turns out to be necessary to compute second partial derivatives of CCC, and this can be quite costly. To see why it's costly, suppose we want to compute all the second partial derivatives ∂2C/∂vj∂vk∂2C/∂vj∂vk\partial^2 C/ \partial v_j \partial v_k. If there are a million such vjvjv_j variables then we'd need to compute something like a trillion (i.e., a million squared) second partial derivatives* *Actually, more like half a trillion, since ∂2C/∂vj∂vk=∂2C/∂vk∂vj∂2C/∂vj∂vk=∂2C/∂vk∂vj\partial^2 C/ \partial v_j \partial v_k = \partial^2 C/ \partial v_k \partial v_j. Still, you get the point.! That's going to be computationally costly. With that said, there are tricks for avoiding this kind of problem, and finding alternatives to gradient descent is an active area of investigation. But in this book we'll use gradient descent (and variations) as our main approach to learning in neural networks.How can we apply gradient descent to learn in a neural network? The idea is to use gradient descent to find the weights wkwkw_k and biases blblb_l which minimize the cost in Equation (6)C(w,b)≡12n∑x∥y(x)−a∥2C(w,b)≡12n∑x‖y(x)−a‖2\begin{eqnarray} C(w,b) \equiv \frac{1}{2n} \sum_x \| y(x) - a\|^2 \nonumber\end{eqnarray}$('#margin_1246306310_reveal').click(function() {$('#margin_1246306310').toggle('slow', function() {});});. To see how this works, let's restate the gradient descent update rule, with the weights and biases replacing the variables vjvjv_j. In other words, our "position" now has components wkwkw_k and blblb_l, and the gradient vector ∇C∇C\nabla C has corresponding components ∂C/∂wk∂C/∂wk\partial C / \partial w_k and ∂C/∂bl∂C/∂bl\partial C / \partial b_l. Writing out the gradient descent update rule in terms of components, we have wkbl→→w′k=wk−η∂C∂wkb′l=bl−η∂C∂bl.(16)(17)(16)wk→wk′=wk−η∂C∂wk(17)bl→bl′=bl−η∂C∂bl.\begin{eqnarray} w_k & \rightarrow & w_k' = w_k-\eta \frac{\partial C}{\partial w_k} \tag{16}\\ b_l & \rightarrow & b_l' = b_l-\eta \frac{\partial C}{\partial b_l}. \tag{17}\end{eqnarray} By repeatedly applying this update rule we can "roll down the hill", and hopefully find a minimum of the cost function. In other words, this is a rule which can be used to learn in a neural network.There are a number of challenges in applying the gradient descent rule. We'll look into those in depth in later chapters. But for now I just want to mention one problem. To understand what the problem is, let's look back at the quadratic cost in Equation (6)C(w,b)≡12n∑x∥y(x)−a∥2C(w,b)≡12n∑x‖y(x)−a‖2\begin{eqnarray} C(w,b) \equiv \frac{1}{2n} \sum_x \| y(x) - a\|^2 \nonumber\end{eqnarray}$('#margin_214093216664_reveal').click(function() {$('#margin_214093216664').toggle('slow', function() {});});. Notice that this cost function has the form C=1n∑xCxC=1n∑xCxC = \frac{1}{n} \sum_x C_x, that is, it's an average over costs Cx≡∥y(x)−a∥22Cx≡‖y(x)−a‖22C_x \equiv \frac{\|y(x)-a\|^2}{2} for individual training examples. In practice, to compute the gradient ∇C∇C\nabla C we need to compute the gradients ∇Cx∇Cx\nabla C_x separately for each training input, xxx, and then average them, ∇C=1n∑x∇Cx∇C=1n∑x∇Cx\nabla C = \frac{1}{n} \sum_x \nabla C_x. Unfortunately, when the number of training inputs is very large this can take a long time, and learning thus occurs slowly.An idea called stochastic gradient descent can be used to speed up learning. The idea is to estimate the gradient ∇C∇C\nabla C by computing ∇Cx∇Cx\nabla C_x for a small sample of randomly chosen training inputs. By averaging over this small sample it turns out that we can quickly get a good estimate of the true gradient ∇C∇C\nabla C, and this helps speed up gradient descent, and thus learning.To make these ideas more precise, stochastic gradient descent works by randomly picking out a small number mmm of randomly chosen training inputs. We'll label those random training inputs X1,X2,…,XmX1,X2,…,XmX_1, X_2, \ldots, X_m, and refer to them as a mini-batch. Provided the sample size mmm is large enough we expect that the average value of the ∇CXj∇CXj\nabla C_{X_j} will be roughly equal to the average over all ∇Cx∇Cx\nabla C_x, that is, ∑mj=1∇CXjm≈∑x∇Cxn=∇C,(18)(18)∑j=1m∇CXjm≈∑x∇Cxn=∇C,\begin{eqnarray} \frac{\sum_{j=1}^m \nabla C_{X_{j}}}{m} \approx \frac{\sum_x \nabla C_x}{n} = \nabla C, \tag{18}\end{eqnarray} where the second sum is over the entire set of training data. Swapping sides we get ∇C≈1m∑j=1m∇CXj,(19)(19)∇C≈1m∑j=1m∇CXj,\begin{eqnarray} \nabla C \approx \frac{1}{m} \sum_{j=1}^m \nabla C_{X_{j}}, \tag{19}\end{eqnarray} confirming that we can estimate the overall gradient by computing gradients just for the randomly chosen mini-batch. To connect this explicitly to learning in neural networks, suppose wkwkw_k and blblb_l denote the weights and biases in our neural network. Then stochastic gradient descent works by picking out a randomly chosen mini-batch of training inputs, and training with those, wkbl→→w′k=wk−ηm∑j∂CXj∂wkb′l=bl−ηm∑j∂CXj∂bl,(20)(21)(20)wk→wk′=wk−ηm∑j∂CXj∂wk(21)bl→bl′=bl−ηm∑j∂CXj∂bl,\begin{eqnarray} w_k & \rightarrow & w_k' = w_k-\frac{\eta}{m} \sum_j \frac{\partial C_{X_j}}{\partial w_k} \tag{20}\\ b_l & \rightarrow & b_l' = b_l-\frac{\eta}{m} \sum_j \frac{\partial C_{X_j}}{\partial b_l}, \tag{21}\end{eqnarray} where the sums are over all the training examples XjXjX_j in the current mini-batch. Then we pick out another randomly chosen mini-batch and train with those. And so on, until we've exhausted the training inputs, which is said to complete an epoch of training. At that point we start over with a new training epoch.Incidentally, it's worth noting that conventions vary about scaling of the cost function and of mini-batch updates to the weights and biases. In Equation (6)C(w,b)≡12n∑x∥y(x)−a∥2C(w,b)≡12n∑x‖y(x)−a‖2\begin{eqnarray} C(w,b) \equiv \frac{1}{2n} \sum_x \| y(x) - a\|^2 \nonumber\end{eqnarray}$('#margin_85851492824_reveal').click(function() {$('#margin_85851492824').toggle('slow', function() {});}); we scaled the overall cost function by a factor 1n1n\frac{1}{n}. People sometimes omit the 1n1n\frac{1}{n}, summing over the costs of individual training examples instead of averaging. This is particularly useful when the total number of training examples isn't known in advance. This can occur if more training data is being generated in real time, for instance. And, in a similar way, the mini-batch update rules (20)wk→w′k=wk−ηm∑j∂CXj∂wkwk→wk′=wk−ηm∑j∂CXj∂wk\begin{eqnarray} w_k & \rightarrow & w_k' = w_k-\frac{\eta}{m} \sum_j \frac{\partial C_{X_j}}{\partial w_k} \nonumber\end{eqnarray}$('#margin_801900730537_reveal').click(function() {$('#margin_801900730537').toggle('slow', function() {});}); and (21)bl→b′l=bl−ηm∑j∂CXj∂blbl→bl′=bl−ηm∑j∂CXj∂bl\begin{eqnarray} b_l & \rightarrow & b_l' = b_l-\frac{\eta}{m} \sum_j \frac{\partial C_{X_j}}{\partial b_l} \nonumber\end{eqnarray}$('#margin_985072620111_reveal').click(function() {$('#margin_985072620111').toggle('slow', function() {});}); sometimes omit the 1m1m\frac{1}{m} term out the front of the sums. Conceptually this makes little difference, since it's equivalent to rescaling the learning rate ηη\eta. But when doing detailed comparisons of different work it's worth watching out for.We can think of stochastic gradient descent as being like political polling: it's much easier to sample a small mini-batch than it is to apply gradient descent to the full batch, just as carrying out a poll is easier than running a full election. For example, if we have a training set of size n=60,000n=60,000n = 60,000, as in MNIST, and choose a mini-batch size of (say) m=10m=10m = 10, this means we'll get a factor of 6,0006,0006,000 speedup in estimating the gradient! Of course, the estimate won't be perfect - there will be statistical fluctuations - but it doesn't need to be perfect: all we really care about is moving in a general direction that will help decrease CCC, and that means we don't need an exact computation of the gradient. In practice, stochastic gradient descent is a commonly used and powerful technique for learning in neural networks, and it's the basis for most of the learning techniques we'll develop in this book.Exercise An extreme version of gradient descent is to use a mini-batch size of just 1. That is, given a training input, xxx, we update our weights and biases according to the rules wk→w′k=wk−η∂Cx/∂wkwk→wk′=wk−η∂Cx/∂wkw_k \rightarrow w_k' = w_k - \eta \partial C_x / \partial w_k and bl→b′l=bl−η∂Cx/∂blbl→bl′=bl−η∂Cx/∂blb_l \rightarrow b_l' = b_l - \eta \partial C_x / \partial b_l. Then we choose another training input, and update the weights and biases again. And so on, repeatedly. This procedure is known as online, on-line, or incremental learning. In online learning, a neural network learns from just one training input at a time (just as human beings do). Name one advantage and one disadvantage of online learning, compared to stochastic gradient descent with a mini-batch size of, say, 202020. Let me conclude this section by discussing a point that sometimes bugs people new to gradient descent. In neural networks the cost CCC is, of course, a function of many variables - all the weights and biases - and so in some sense defines a surface in a very high-dimensional space. Some people get hung up thinking: "Hey, I have to be able to visualize all these extra dimensions". And they may start to worry: "I can't think in four dimensions, let alone five (or five million)". Is there some special ability they're missing, some ability that "real" supermathematicians have? Of course, the answer is no. Even most professional mathematicians can't visualize four dimensions especially well, if at all. The trick they use, instead, is to develop other ways of representing what's going on. That's exactly what we did above: we used an algebraic (rather than visual) representation of ΔCΔC\Delta C to figure out how to move so as to decrease CCC. People who are good at thinking in high dimensions have a mental library containing many different techniques along these lines; our algebraic trick is just one example. Those techniques may not have the simplicity we're accustomed to when visualizing three dimensions, but once you build up a library of such techniques, you can get pretty good at thinking in high dimensions. I won't go into more detail here, but if you're interested then you may enjoy reading this discussion of some of the techniques professional mathematicians use to think in high dimensions. While some of the techniques discussed are quite complex, much of the best content is intuitive and accessible, and could be mastered by anyone. Implementing our network to classify digitsAlright, let's write a program that learns how to recognize handwritten digits, using stochastic gradient descent and the MNIST training data. We'll do this with a short Python (2.7) program, just 74 lines of code! The first thing we need is to get the MNIST data. If you're a git user then you can obtain the data by cloning the code repository for this book,git clone https://github.com/mnielsen/neural-networks-and-deep-learning.git If you don't use git then you can download the data and code here.Incidentally, when I described the MNIST data earlier, I said it was

Older spouses may be more mature, but later marriage has its own challenges.

When school administra-tors take away students’ phones or tell them to put them away during class time, they are teaching implicit lessons about the kind of work environments these students are expected to en-ter. In this light, digital literacies are a matter of social justice

But while the urban tribe helps us survive, it does not help us thrive. The urban tribe may bring us soup when we are sick, but it is the people we hardly know—those who never make it into our tribe—who will swiftly and dramatically change our lives for the better.

Each time we are handed any piece of literature, we are given a story made up of different features. This may include elements like genre, audience, and purpose. Genre provides the reader with an idea about what will happen as they read. It will also provide some sort of background, timeline, and even give into the storyline just on the basis of what the author has chosen the genre of their piece to be. There are many different types of genre including romance, mystery, and even thriller. The audience is the group of people that the author is trying to convey their message to, or in general just the people who they want to reach. The message is specific to the audience and I think genre plays a part in who your audience will be. If you’re writing a fantasy novel, those who enjoy reading fantasy will be more likely to read it. However, if you’re writing a non-fiction piece, your audience will most likely be different. And the list continues, as many people enjoy reading different types of literature. So as I mentioned, genre and even purpose may play a role in who your audience will be, but the message is what exactly and who exactly will make up your “audience”. Purpose is the point you are trying to make, it is one’s way of expression and or presentation of information. This is the author’s way of communicating with their audience.

Our primary priority should neither be minimizing cost nor maximizing pedagogical flexibility. Our primary priority should be increasing student learning, and our efforts to reduce costs and increase pedagogical flexibility must always be subservient to that end. When we fail to put student learning first, we can become zealots who confuse the means with the ends.

Express.co.uk

Express.co.uk

Express.co.uk

Express.co.uk

Like the national debt, the homeless population, gun ownership, and job insecurity, television grew prodigiously in the 1980s.

However, the artists of West Meets East do not possess institutional power, and are instead most likely more suppressed by institutional norms of boundary and difference that often accompany immigration.

Daily Beast

Since I weeded out the folks that don’t teach their kids logic in the first two paragraphs, as representatives of the real world it’s up to the rest of us to spread the knowledge. It won’t be easy. The best thing we can do is teach these thought processes to our children, so that they may look at other children with looks of bewilderment when other children are unable to solve simple tasks. Hopefully, they will not simply do the task for them, but teach them to think. I’m not saying we need to build a whole new generation of project managers and analysts, but it would be better than a generation of task-oriented mindless office drones with untied shoelaces, shoving on a door at the Midvale School for the Gifted.

to countries without regard to the recipients’ needs or capacity.

to maintain a status-quo wherever we find existing regimes anti-communistic.”

“Uncle Sugar is as dangerous a role for us to play as Uncle Shylock.”

the East may think little or nothing of them.”

Can you show that the test error is an unbiased estimate of the generalization error? That is, that the expected value of the test error is equal to the generalization error.

First, the self may be seen as a social actor, who enacts roles and displays traits by performing behaviors in the presence of others.

What is your story? What kind of a narrative are you working on? As you look to the past and imagine the future, what threads of continuity, change, and meaning do you discern? For many people, the most dramatic and fulfilling efforts to change the self happen

, the self may be seen as a social actor, who enacts roles and displays traits by performing behaviors in the presence of others

e students may learn that compliance is important and that certain behaviors will be academically rewarded and allowed to compensate for learning deficiencies. On the other hand, instructional strategies such as project-based learning or community-based learning, to name just two of many possible options, may communicate specific messages

However, many of us are not fully aware of our values, and as a result can at times think, say and do things that are not in alignment.

It represents the smallest complete television picture umt. Bu~ images already are combinations of several different signs at once and mvolve a complex set of denotations and connotations. Further-more, if we use the frame as the smallest unit of meaning, we ignore the soundtrack, where ½o second would not necessarily capture a meaning-~l sound and where speech, sound effects, and music may be occurring simultaneously.

Extrapolating our results to the general population, we estimate that asking about citizenship would reduce the share of Hispanics recorded by the Census by approximately 6.07 million, or around 12.03 percent of the 2010 Hispanic population – a sizable reduction in the share of the U.S. population that would be recorded as Hispanic.

We think what they want is flexibility, but actually what they need is structure

One would think that if there's play in the joints, that that would include the concept that States are free to say we don't want to spend money from the public FIs on houses of worship. Now, you say this affects free exercise. We seem to be confusing money with religious practice.

The authors make the point in ch4 that annotation is not really conversation. I think Miranda is correct here, that annotation is a work-around for the fact that texts are not interactive. Ultimately, are we just fooling ourselves by annotating a text?

This intangible nar-rating presence need not be thought of as a person, but rather as an agency, that which chooses, orders, presents, and thus tells the narrative before us.

how do we get students to move beyond civil attention to true engagement in the classroom?

writeandanalyzeavarietyoftypesof

How do we define a composing practice that is in-terlaced and interwoven with email, text-messaging, and web-browsing?

Husmann thinks the students had fallen into certain study habits, which, once formed, were too hard to break. Students seemed to be interested in their learning styles, but not enough to actually change their studying behavior based on them. And even if they had, it wouldn’t have mattered.

Be it further enacted, that every civil officer shall, and every person may, arrest and carry back to his or her legal employer any freedman, free Negro, or mulatto who shall have quit the service of his or her employer before the expiration of his or her term of service without good cause, and said officer and person shall be entitled to receive for arresting and carrying back every deserting employee aforesaid the sum of $5, and 10 cents per mile from the place of arrest to the place of delivery, and the same shall be paid by the employer, and held as a setoff for so much against the wages of said deserting employee: Provided, that said arrested party, after being so returned, may appeal to a justice of the peace or member of the board of police of the county, who, on notice to the alleged employer, shall try summarily whether said appellant is legally employed by the alleged employer and his good cause to quit said employer; either party shall have the right of appeal to the county court, pending which the alleged deserter shall be remanded to the alleged employer or otherwise disposed of as shall be right and just, and the decision of the county court shall be final.

facts toward insight or complex ideas rarely take place” in many classrooms.By the same token, if we’re still using tests (Kohn, 2015) rather than authentic assessments of theirunderstanding, we’re giving them a gift of meaningful questions with one hand only to take it awaywith the other (by evaluating them on the number of facts they’ve crammed into short-termmemory).Deep questions help kids to stay curious, grow increasingly resourceful at figuring things out, andbecome active meaning makers. To structure learning around such questions is to take the first steptoward creating an environment that is not merely academic but genuinely intellectual.2. WHOSE QUESTIONS?It makes good sense to create thoughtful questions for students, but it’s even more important toelicit their questions — a possibility missing from many resources on the subject. Teaching, likeparenting and managing, is greatly improved by following a four-word admonition: Talk less, askmore. And better than asking subject-matter questions is the process of encouraging kids to come upwith the questions that matter to them. If this is more the exception than the rule in our classrooms,it may be because it requires us to give up some control.There is a purely practical justification for asking students what they’re curious about: Even amarvelously gifted teacher can’t always figure out the right question to ask a given student at justthe right time, as Duckworth (1987, p. 5) pointed out. Thus, she added, it’s fortunate that “childrencan raise the right question for themselves if the setting is right.” As a result, “they are moved to taxthemselves to the fullest to find an answer.” By inviting their questions, we unleash the power ofintrinsic motivation because all of us tend to develop more enthusiasm for, and ultimately becomemore skillful at, pursuing questions about which we’re genuinely curious than those handed to us bysomeone else.This may explain the National Research Council’s (1996, p. 31) declaration that “inquiry intoauthentic questions generated from student experiences is the central strategy for teachingscience.” (By “is,” we can assume they meant “should be.”) And their conclusion is by no meanslimited to science.In more traditional classrooms, students are rarely asked what they’ve been wondering abou

Cognitive–Behavioral Therapy: A problem-focused intervention that seeks to teach chil-dren who manifest socioemotional challengesand maladaptive behaviors to become theirown therapist.

that will empower you to advocate for yourself, make smart choices for your own learning, and get the most out of your experience at SF State and the culture we collectively create here.

does not exist in this province.

Express.co.uk

The question posed at the beginning of this paragraph is a good place to begin.

We are usually relatively unfamiliar with our participants, at least on a personal level. This can make sitting down for an interview where we might be asking some deep questions a bit awkward and uncomfortable, at least at first. Because of this, we want to craft our questions in such a way that they are not off-putting, inadvertently accusatory or judgmental, or culturally insensitive.  To accomplish this we want to make sure we phrase questions in a neutral tone (e.g. “Tell me what that was like”, as opposed to, “That sounds horrible, what was that like”). To accomplish this we can shift perspectives and think about what it would be like for us to be asked these questions (especially by a stranger), and we can pilot test our questions to see how they ‘feel’ to others. Also, if we are conducting interviews on topics that may be particularly hard for people to talk about, we likely will want to start out with some questions that are easier to address prior to getting into the heavier topics. Make them relatable Unlike surveys, where researchers may not be able to explain the meaning of question, when conducting interviews, we are present to help further explain questions if there is some confusion. However, ideally our questions are as clear as possible from the beginning. This means that we avoid jargon or technical terms, we anticipate areas that might be hard to explain and try to provide some examples or a metaphor that might help to get the point across, and we do our homework to relay our questions in a cultural context that is appropriate. Like the discussion above, pilot testing our questions can be very helpful for ensuring the relatability of our questions, especially with community representatives. What sounds good in our heads as a question, might make little sense to our intended audience. Make them individually distinct, but collectively comprehensive Just like when we are developing survey questions, you don’t want to ask more than one question at the same time. This is confusing and hard to respond to for the participant, so make sure you are only asking about one idea in each question.  However, when you are thinking about your list of questions, or your whole interview guide collectively, ensure that you have comprehensively included all the ideas related to your topic. It’s extremely disheartening as a qualitative researcher that has concluded their interviews and realized there was a really important area that you failed to include in your guide. To avoid this, make sure to know the literature in your area well and talk to other people who study this area to get there perspective on what topics need to be included.

you may choose to meet at your office

It is a resource that participants own that they choose to share with us.

As we are thinking about going out in the world to gather data, I think it can be helpful to think about the data that is shared with us a resource.

Daily Beast

Emergent design is the idea that some decision in our design will be dynamic and change as our understanding of the research question evolves as we go through the research process.

The researcher who schedules interviews with public assistance recipients to capture their experience after a legislation drastically changes their requirements for receiving benefits relies on the verbal data shared with them.

As such, as you go about recruiting for your qualitative studies, remember that people are made of multiple stories, of intersectional identities.

Because of this, I think we need to take special care to treat these stories as sacred and we go about asking for people to share them, we need to do so humbly.

assure there is fair distribution of risks and benefits related to our research, be conscientious in our recruitment efforts to support equitable representation, and that we ensure special protections to vulnerable groups involved in research activities. As you plan your qualitative research study sampling plan, make sure to consider who is invited and able to participate and who is not.

[2.20]

Section 2 then begins the real ball game, namely everyone has the following fundamental freedom. The one difficulty we had, as a committee, is with Section 2(b). What do we do with freedom of thought when you have got legislation dealing with have propaganda? How far is it possible to retain such articles as Section 281(1) of the Criminal Code and Section 281(2)? Moreover, you will see we have quoted from Article 20 of the United National Covenant of Civil and Political Rights where propaganda of this kind is regarded as inconsistent with freedom of speech. [Page 87] So we raised the question which seemed to me to be necessary to raise with you, that caution must be exercised, we hope, by the courts in due course, or by you, as draftsmen on how far you are prepared to push the concept of free speech consistent with our experience of hate propaganda. One suggestion we make here-and I do not wish to do anything more than to drop it as a hint, but you may want to have some language that some of the modern constitutions have, which state very starkly and flatly that the advocacy of genocide or group libel is forbidden. But I had the honour to be the chairman of the special committee on hate propaganda in 1965. At that time we came to the flat conclusion that the advocacy of group hatred and genocide was totally inconsistent with the democratic process and no democratic state could tolerate it. Now, whether you want to put that flatly in a constitution is for you to consider; but I think it is for us to bring it to your attention, because it is of importance.

Is Eugenics Happening Today? As research continues to uncover new disease-causing mutations, it becomes increasingly possible to stop the transmission of certain heritable diseases.  In the long term, this may lead to complete eradication of diseases like Down Syndrome, cystic fibrosis, and hemophilia. However, some wonder if modern day attempts to eradicate hereditary disorders equate to eugenics. One complication of genetic testing for the purpose of disease eradication is that, in practice, a particular ethnic group will likely be involved due to shared ancestry.  For instance, Tay-Sachs disease is significantly more common in certain Jewish communities.  Tay-Sachs is a genetic disease that causes a deterioration of mental and physical abilities and results in death by age four. Eradicating Tay-Sachs will require screening all individuals in the affected population. However, a public campaign to test all individuals of Jewish descent for Tay-Sachs carrier status may for some recall the racist motivations of eugenicists in the early 20th century, particularly those associated with Nazi Germany. Also, racial stereotypes or biases may be reinforced if genetic testing performed on individuals of an ethnic group reveals a predisposition to a particular disease or condition. Using modern genetic technology, prospective parents can be prescreened to determine their carrier status for certain diseases. Preimplantation genetic diagnosis following in vitro fertilization allows parents to select embryos that are free of disease. Additionally, prenatal genetic testing can provide a lot of information to parents about their unborn child.  These technologies make more informed decision-making possible, but some are concerned about a shift in the way we view family and parenting.  Parents who want to have a child without pursuing genetic testing may feel guilty if the child is born with any health problems. Additionally, some are concerned about what an overemphasis on eliminating disabilities in unborn children will mean for people who already have the disability. The most significant difference between modern genetic technologies, that some view as eugenic, and the historical use of eugenics is consent.  Today, individuals pursue genetic testing by choice.  An individual can never be forced into testing or be required to take action, such as sterilization, based on the results of a genetic test.  Individuals differ in their views on genetic testing in relation to reproductive decision-making and possible eugenic motivations, but at least today parents have the choice to use the technology or not.

I want to refer you to Section 2 of the resolution which is a Section on fundamental freedoms. It says: Everyone has the following fundamental freedoms: (a) freedom of conscience and religion, (b) freedom of thought, belief, opinion expression, including freedom of the press and other media of information What I want to ask you is, how do you think the word “everyone” would be interpreted as it pertains to everyone has the following freedoms, the freedom of the press, freedom of other media of information. I want to take you back in this country about four of five years when the government across the way introduced legislation, which I supported, concerning Time magazine and Reader’s Digest, to try and Canadianize the magazine industry in this country. I am wondering whether or not if we were to enshrine Section 2 in the constitution as written, Time magazine or Reader’s Digest could have gone to the courts and said: “We have a consitutional right in this country of freedom of expression and freedom of the press and freedom of information, freedom of the media; therefore, the government of Canada [Page 12] and the Parliament of Canada do not have the right to legislate restrictively against our two organizations.” Could it be interpreted in that way? Ms. Crandall: Mr. Nystrom, I think that is the kind of question which an expert should be asked to answer. This is what we are saying now, We have not had an opportunity to look at all sides of these questions to give you any kind of an answer. Again, I am not trying to be difficult. But that is one of the questions which we would like to ask someone who is knowledgeable. Mr. Nystrom: I appreciate the answer. The reason why I ask the question is that the words “everyone” and “citizens of Canada” are used throughout the resolution. I am not a lawyer myself, but it would seem to imply that these could be given a fairly wide interpretation, and I am concerned that we might have in a constitution something that is restrictive where we could not increase Canadian content. Let me ask you the same question again about the electronic media. There is growing concern that we Canadianize radio, television—and the CRTC is concerned about this, about television programs coming in from the United States. There is talk now about a second CBC network in this country. Again, I want to ask you a similar question pertaining to the electronic media. If everybody has the freedom of expression and freedom of the press and other media of information, in your opinion, or perhaps in the opinion of your colleague, do you think we would be able to do this as a Parliament, where the constitution says we are denying a fundamental right to everyone, perhaps NBC, New York, or ABC somewhere in the United States? Ms. Hardy: I think, Mr. Nystrom, that it is very important. I have served abroad for Canada in the Department of External Affairs, in the public affairs field, and I feel that it is very important that we develop a Canadian culture, that we develop an interest in things Canadian and a pride, and I grant that there are very good programs produced by the electronic media of other countries but I think we should be proud of our own heritage and be proud of what we can do. I have just been at a briefing on plans for CBC 2, Tele Deux, and I am very pleased that this is what may be coming along shortly and I would hope that we would not refuse all foreign media offers to assist us in our cultural development, but I think we should certainly give ourselves the chance to be first in the field and to welcome the opportunity and the pride in our own country and in what we can develop ourselves. This is a continuing subject of interest financially as well as culturally, naturally, and I would hope that the media club, which now covers the electronic media representatives as well as the press, would be in the forefront of assisting in developments if possible. Thank you. [Page 13] Mr. Nystrom: I wonder if you could possibly, if you have time to do a written brief to the Committee, to try and seek some advice on those questions, because I agree fully with you that we have to develop a Canadian culture and of course we need some input from other countries around the world because we are part of the global village, we have to have a Canadian identity and it is very important, and I would be very concerned if the way Section 2 is written that perhaps we could be denied through our constitution the right to develop fully the Canadian culture and pehaps you could look at that. I also wanted to ask your interpretation of a couple of other words in Section 2. I wanted to ask you what you think the interpretation in your opinion would be of other media of information. We have singled out here freedom of belief, opinion, expression, including the freedom of the press. I know what the press is, I think, but what would be the interpretation legally, in your opinion, of other media of information, what would that include? Ms. Hardy: I would expect that that would include the electronic journalism. The press is usually referred to as print media. Media is a very broad term that has had to be used because you cannot just refer to the press now because it covers a number of other representatives who inform, through one source or another, and I think the electronic media has an important place now in our culture because communications in this country is an aspect of helping unify the country, I think, by letting us get to know each other, not only through print but through electronic means.

and the more pseudonymity is available

Yet the body intervenes constantly, whether one is ill or not. It is the mode of intervention that conditions how well, or unwell, we feel. A state of wellbeing is one in which we do not need to think about our embodied organism in any way other than the sensorial pleasures it affords, where we are immersed within our environment, engaged in an activity, involved with others. But one of physical or emotional pain affects the very foundation on which the sense of self we otherwise take for granted rests: what we feel ourselves to be can be upended. When this happens, we may realise that what we feel ourselves to be is in fact constructed. How we exist as embodied selves is a highly complex business involving the brain and body engaged in constant interaction.

How will this class, with its focus on biological organic chemistry, be different from a class in biochemistry? You should not make the mistake of thinking that a course built around biological examples is going to be more biochemistry than organic chemistry. Although there will inevitably be some overlap with the biochemistry course you may take later in your career (just as there is overlap between courses in general chemistry and organic chemistry, between physical chemistry and physics, etc.), this is very much an organic chemistry text – it follows essentially the same outline and covers essentially the same topics as most other organic chemistry texts, and is fundamentally distinct from most biochemistry texts. In organic chemistry, we think about individual chemical reactions at the atomic level – concentrating on the 'how and why' details of what is happening with the organic molecule(s) in question. In a biochemistry course, you will see many of the same compounds and reactions that you saw in this text. In biochemistry, however, you will see them mainly from the perspective of complete biochemical pathways: you will spend much of your time studying how these pathways are interconnected and regulated. Biochemistry, then, is a 'bigger picture' course where we study the forest, while in organic chemistry we study the trees.

Our greatest danger is that in the great leap from slavery to freedom we may overlook the fact that the masses of us are to live by the productions of our hands, and fail to keep in mind that we shall prosper in proportion as we learn to dignify and glorify common labour, and put brains and skill into the common occupations of life;

This triple paradox in Mr. Washington's position is the object of criticism by two classes of colored Americans. One class is spiritually descended from Toussaint the Savior, through Gabriel, Vesey, and Turner, and they represent the attitude of revolt and revenge; they hate the white South blindly and distrust the white race generally, and so far as they agree on definite action, think that the Negro's only hope lies in emigration beyond the borders of the United States. And yet, by the irony of fate, nothing has more effectually made this programme seem hopeless than the recent course of the United States toward weaker and darker peoples in the West Indies, Hawaii, and the Philippines,—for where in the world may we go and be safe from lying and brute force?

reets; for if we take the ages into our account, may there not be a civilization going on among brutes as well as men? They seemed to me to be rudimental, burrowing men, still standing on their defence, awaiting their transformation.

I wonder, though, why it was thought that the appropriate location for a limitations clause was right at the very beginning of the proposed Charter of Fundamental Rights and Freedoms, and whether you would be prepared to look at the possibility of moving the limitations clause in whatever form we may finally end up with—and naturally, there may be complications, because if there is a preamble in it it would be inappropriate to do this; but if there is no preamble, on the proposed Clause 1, moving the limitations clause to the end of the proposed Charter? In other words, I believe it would be important symbolically, if nothing else, to start out with the list of fundamental freedoms, the mobility rights and a positive statement of what the rights are, and then at the conclusion of the proposed Charter to indicate what limitations might exist on those [Page 26] rights, rather than starting out with limitations and then a statement of rights. Mr. Kaplan: Well, before directly answering the question, I would like to understand that you are suggesting that the difference is symbolic, and that it does not make any real difference. Mr. Robinson: Certainly, I would not argue that there was any difference in substance in the way this will be interpreted, and I am sure your advisers could confirm that. Mr. Kaplan: I do not think there any real difference either. I think it is purely a matter of style. Our view of the matter was that it was more realistic and useful to the reader to see at once that the rights were not absolute, but that they were constrained. That would be made immediately clear to a person consulting the statute without having to read the first 30 sections to find that what was contained in the proposed Clause 1 was 0really not the whole story. It seems to be more honest. Mr. Robinson: Mr. Minister, if it is agreed that there is no difference in substance—and we are talking about a document, a Charter of Rights which, hopefully, would be widely distributed to school children, to Canadians right across the country, and I would suggest there is a certain symbolic value in setting out those rights and then at the conclusion of those rights indicating what the limitations, if any, may exist in respect of them, as is done in the Diefenbaker Bill of Rights, where the rights were enumerated and at the conclusion of the Bill of Rights, there are references, for example to the War Measures Act and to other limiting provisions.

Mr. Roger Tassé, (Q.C., Deputy Minister, Department of Justice): I think that is an important question you have raised, Mr. Fraser. In effect when you look at the meaning of law, it may mean a number of things and in this context it could mean an Act of Parliament, for example, and we did not want it to be restricted to an Act of Parliament for some of the reasons that have been expressed, and also for another reason that has not been mentioned so far, and that is in effect we wanted also to cover rules of the common law. For example, in the area of libel, defamation. And in many provinces this has not been clarified. There are rules that have just been expressed over time by the courts and we did not want to upset all of this legislation so that is why in effect in French we have used an expression that would embody as well rules of common law that have been established by courts and it could be in the civil law field or in the common law, most probably in the common law, but also would include the statute and include a regulation enacted under an appropriately passed or enacted legislation. Mr. Fraser: So what you are saying, then, is that … Mr. Tassé: Perhaps if I may just expand on what I have just said. For example, if you look at the freedom of expression, the law of defamation, the law of libel imposes some limits on that so we wanted these to continue to have application and we think that they would fit in effect the tests that are set out in Section 1. Mr. Fraser: Well, then, by the same token, so does the law of master and servant, the law of contract and the law of partnership, and a number of other common law notions. [Page 50] Could you foresee a situation where, on the basis of the rights set out in here, you could have a conflict between what are considered laws which stem from the body of case law that has come down over the centuries which could be in conflict with the right that has been set out in the Charter? Mr. Tassé: Well, Mr. Fraser, we do not see these rights or these prescriptions of the Charter to have application in terms of a relationship between individuals. We see them as applying in terms of a relationship between the state and individuals, so I am not sure that in terms of contract laws, unless we were looking at the situation where in fact we are talking of contracts passed between the state, the government, and that might offend a constitutional limitation on some of these rights, then the Charter might be called upon for assistance but if we are just looking at in effect relationships, contractural relationships between individuals, I do not see how the Charter itself could be called upon to assist in resolution of conflicts that may arise. Mr. Fraser: Well, I do not want to take this too far into the realm of theory but individual contracts are constantly formed as a result of discrimination between certain options and certain individuals, and that has always been, within some limitations, an accepted freedom to enter into contract unless there is a specific piece of legislation which forbids it. You can take, for instance, the codes in some of the provinces which now constrain absolute freedom of contract in hiring policies. But I take it that what you are saying is that in the English version when you say “prescribed by law”, that is not just statute law, but is also the common law? Mr. Tassé: Yes. Mr. Fraser: As decided by the cases? Mr. Tassé: Yes. Mr. Fraser: But they could still be challenged if somebody could take the issue to a court and say that that law can no longer be demonstrably justified in a free and democratic society? Mr. Tassé: That is correct. Mr. Fraser: Thank you.

Mr. Irwin: Is that not the whole raison d’être of this discussion, how much power are we as legislators going to give up by way of entrenched rights to the courts, to not be touched forever. The Joint Chairman (Mr. Joyal): Mr. Tarnopolsky. [Page 27] Professor Tarnopolsky: Mr. Chairman, if I may answer that. I think our position is that you should either fish or cut bait. If you are going to have a bill of rights, make it a bill of rights which cannot be just over-ridden any time that a court is convinced, which we are suggesting would be relatively easy, that the limits are those which are generally acceptable. It would not just be in legislation, because, again, if I could use the Hogan case, I think that it would not be very difficult to convince a court that the practice of the police, namely, how can a lawyer help the chap, let him take the breathalizer, it is probably generally accepted, So that I think the fear of those of our members who support a bill of rights would be that it would be disillusioning, that it would be disappointing for the populace to think that they have a bill of rights which really over-rode inconsistent legislative administrative action to find out it is not. We think that rather than promoting that kind of cynicism, the Parliament should face up to either creating a bill of rights which over-rides or stay with the one which we now have, which we have got some jurisprudence on and there are more cases than just a Drybones case which have been applied with some effect.

Mr. Fairweather, what I would like to do from looking at materials that you have provided earlier, is take you to your concern of Section I of the proposed resolution. At the bottom of your presentation as well as in the accompanying documents you stress your concern about Section l. I would like to ask you from your perspective if the clause remains essentially in the form it now appears what are the technical consequences of that clause in relation to the protection of rights and freedoms? Mr. Fairweather: They are so serious that I could not imagine this Committee letting Section l go unamended, That section as drafted would challenge, in my opinion, the rest of the charter, and I suspect somebody is going to be getting an amendment. It is, as I said, turning our backs on the international and national jurisprudence, and it is very broadly drafted. Why we do not use the language that is well accepted now and has been ratified by Canada, for the life of me I have no idea. This is a strong statement but I have strong feelings. I am absolutely committed to the entrenchment and the patriation and the goals. They are wonderful goals for this country at last but why not go for something better. Mr. Epp: From your experience as a Commissioner of the Human Rights Commission, could you give us examples of if this proposed resolution had in fact been in effect with the prohibitions in Section 1, can you give us some specific examples of the restrictions it would have given or caused both to rights and freedoms and also to the Commission? Mr. Fairweather: Yes. One was given last night, if I know correctly, by the Minister of Justice for Canada. It might be that generally accepted standards in this country for mandatory retirement, the anti-discrimination part having to do with age, could be challenged and rendered meaningless as a reform mechanism, because the generally accepted standards now are quite illiberal, if I may use that word in this place. [Page 12] The generally accepted standards for Canada are to push people out at certain ages, I greeted this charter with excitement when I saw that the Government of Canada had included age, but when I see the language of Section 1, I wonder. Another message that surely cannot be forgotten is that the generally accepted standards in Canada in 1940 and 1941 were to take Canadians whose offence was that they were of Japanese origin. In the Wellington Street Archives last night while you were doing something else I went to the festival of the 100th anniversary of the Chinese in Canada. They came to build the railway and I am as shocked as I know senators and members are to remember that people who are enshrined and rightly, in our history were perpetrators of the Chinese Exclusion Act, the Chinese $50 a head tax act. The Chinese people in this country were not allowed to vote in a province I think until after the Second World War. It was Mr. Diefenbaker who gave the Native peoples the right to vote in the late 1950s. I am not saying that a government that follows this one would, but it could, because those were I guess until reform came the accepted standards. This document is to be entrenched, enshrined, as a statement to the people who have their loyalty to Canada. Mr. Epp: Do I understand you correctly, sir, that if the document before us were to be enshrined, entrenched, that an incident like Canadians of Japanese origin and the removal of these people from various parts of Canada, their prohibition of freedoms at that time, that in fact that kind of action would still be possible under the charter in its present form. Mr. Fairweather: I think, Mr. Epp, my duty is to warn you, and I have given some examples, age and these other offensive matters could be put in jeopardy. I am not saying they will but they could. I think most of this charter is really a superb piece of work, but I cannot see why Canada wants to turn its back on accepted international standards and language that has been adjudicated. That is why I am being a bit fussy. It clouds the rest of a noble document.

in May, an Australian think tank called climate change "a near- to mid-term existential threat to human civilization."