hduser@benjamin-VirtualBox:/usr/local/hadoop$ /home/hduser/hive/bin/hive 13/12/17 15:35:40 INFO Configuration.deprecation: mapred.input.dir.recursive is deprecated. Instead, use mapreduce.input.fileinputformat.input.dir.recursive 13/12/17 15:35:40 INFO Configuration.deprecation: mapred.max.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize 13/12/17 15:35:40 INFO Configuration.deprecation: mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize 13/12/17 15:35:40 INFO Configuration.deprecation: mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.rack 13/12/17 15:35:40 INFO Configuration.deprecation: mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.node 13/12/17 15:35:40 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces 13/12/17 15:35:40 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative Logging initialized using configuration in jar:file:/home/hduser/hive/lib/hive-common-0.12.0.jar!/hive-log4j.properties SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/hduser/hive/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] hive> C CAST CLUSTER CLUSTERED COLLECTION COLUMNS COMMENT CREATE hive> CREATE TABLE TABLE TABLES TABLESAMPLE hive> CREATE TABLE foreign_aid > (country STRING, sum BI BIGINT BINARY > (country STRING, sum BIGINT ) > ROW FORMAT DELIMITED > FILEDS TERMINATED BY ',' > STORED AS TE TEMPORARY TERMINATED TEXTFILE > STORED AS TEXTFILE ; NoViableAltException(26@[1518:103: ( tableRowFormatMapKeysIdentifier )?]) at org.antlr.runtime.DFA.noViableAlt(DFA.java:158) at org.antlr.runtime.DFA.predict(DFA.java:144) at org.apache.hadoop.hive.ql.parse.HiveParser.rowFormatDelimited(HiveParser.java:22901) at org.apache.hadoop.hive.ql.parse.HiveParser.tableRowFormat(HiveParser.java:23091) at org.apache.hadoop.hive.ql.parse.HiveParser.createTableStatement(HiveParser.java:4388) at org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:2016) at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:1298) at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:938) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:190) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:424) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:342) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:977) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:888) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:781) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) FAILED: ParseException line 4:0 cannot recognize input near 'FILEDS' 'TERMINATED' 'BY' in serde properties specification hive> > > > > STORED AS TEXTFILE ; [7]+ Stopped /home/hduser/hive/bin/hive hduser@benjamin-VirtualBox:/usr/local/hadoop$ hduser@benjamin-VirtualBox:/usr/local/hadoop$ hduser@benjamin-VirtualBox:/usr/local/hadoop$ hduser@benjamin-VirtualBox:/usr/local/hadoop$ /home/hduser/hive/bin/hive 13/12/17 15:37:29 INFO Configuration.deprecation: mapred.input.dir.recursive is deprecated. Instead, use mapreduce.input.fileinputformat.input.dir.recursive 13/12/17 15:37:29 INFO Configuration.deprecation: mapred.max.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize 13/12/17 15:37:29 INFO Configuration.deprecation: mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize 13/12/17 15:37:29 INFO Configuration.deprecation: mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.rack 13/12/17 15:37:29 INFO Configuration.deprecation: mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.node 13/12/17 15:37:29 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces 13/12/17 15:37:29 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative Logging initialized using configuration in jar:file:/home/hduser/hive/lib/hive-common-0.12.0.jar!/hive-log4j.properties SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/hduser/hive/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] hive> > > CREATE TABLE foreign_aid > (country STRING, sum BIGINT ) > ROW FORMAT DELIMITED > FIELDS TERMINATED BY ',' > STORED AS TEXTFILE ; OK Time taken: 29.013 seconds hive> > SHOW TABLES; OK foreign_aid Time taken: 0.32 seconds, Fetched: 1 row(s) hive> DESCRIBE format format format_number( hive> DESCRIBE foreign_aid; OK country string None sum bigint None Time taken: 0.153 seconds, Fetched: 2 row(s) hive> LOAD DAT DATA DATE DATETIME hive> LOAD DATA INP INPATH INPUTFORMAT hive> LOAD DATA INPATH 'foe > re > ; MismatchedTokenException(26!=286) at org.antlr.runtime.BaseRecognizer.recoverFromMismatchedToken(BaseRecognizer.java:617) at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115) at org.apache.hadoop.hive.ql.parse.HiveParser.loadStatement(HiveParser.java:1414) at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:1253) at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:938) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:190) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:424) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:342) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:977) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:888) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:781) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) FAILED: ParseException line 1:18 mismatched input 'foe' expecting StringLiteral near 'INPATH' in load statement hive> LOAD DATA INPATH 'ForeignAssistanceData.csv' > OVERWRITE INTO TABLE foreign_aid; Loading data to table default.foreign_aid Table default.foreign_aid stats: [num_partitions: 0, num_files: 1, num_rows: 0, total_size: 100234, raw_data_size: 0] OK Time taken: 0.702 seconds hive> SELECT * FROM foreign_aid LIMIT 10; OK Afghanistan 314552000 Afghanistan 1200000 Afghanistan 400000000 Afghanistan 1176000 Afghanistan 2400000 Afghanistan 650000 Afghanistan 1500000 Afghanistan 23000000 Afghanistan 1100000 Afghanistan 40000000 Time taken: 0.432 seconds, Fetched: 10 row(s) hive> SELECT country, SUM(sum) FROM foreign_aid GROUP BY country; Total MapReduce jobs = 1 Launching Job 1 out of 1 Number of reduce tasks not specified. Estimated from input data size: 1 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer= In order to limit the maximum number of reducers: set hive.exec.reducers.max= In order to set a constant number of reducers: set mapred.reduce.tasks= SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/hduser/hive/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 13/12/17 15:43:06 WARN conf.Configuration: file:/tmp/hduser/hive_2013-12-17_15-43-02_056_7905750711769655792-1/-local-10002/jobconf.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring. 13/12/17 15:43:06 WARN conf.Configuration: file:/tmp/hduser/hive_2013-12-17_15-43-02_056_7905750711769655792-1/-local-10002/jobconf.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring. 13/12/17 15:43:06 INFO Configuration.deprecation: mapred.input.dir.recursive is deprecated. Instead, use mapreduce.input.fileinputformat.input.dir.recursive 13/12/17 15:43:06 INFO Configuration.deprecation: mapred.max.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize 13/12/17 15:43:06 INFO Configuration.deprecation: mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize 13/12/17 15:43:06 INFO Configuration.deprecation: mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.rack 13/12/17 15:43:06 INFO Configuration.deprecation: mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.node 13/12/17 15:43:06 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces 13/12/17 15:43:06 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative Execution log at: /tmp/hduser/.log Job running in-process (local Hadoop) Hadoop job information for null: number of mappers: 0; number of reducers: 0 2013-12-17 15:43:11,508 null map = 0%, reduce = 0% 2013-12-17 15:43:15,225 null map = 100%, reduce = 100% Ended Job = job_local1229925622_0001 Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin OK Afghanistan 5398420000 Africa Regional Office (DOS) 160550000 Africa Regional Office (USAID) 231685000 African Union (DOS) 1760000 Albania 45367000 Algeria 13744000 Ambassador-at-Large for Global Women’s Issues (DOS) 5000000 Angola 132593000 Argentina 2745000 Armenia 105965000 Asia Middle East Regional Office (USAID) 52980000 Azerbaijan 63004000 Bahrain 37254000 Bangladesh 470835000 Barbadas 841000 Barbados and Eastern Caribbean (DOS & USAID) 73522000 Belarus 22072000 Belize 6374000 Benin 58015000 Bolivia 61539000 Bosnia and Herzegovina 99999000 Botswana 138869000 Brazil 36845000 Bulgaria 22851000 Bureau for Democracy NULL Bureau for Economic Growth NULL Bureau for Energy Resources (DOS) 9000000 Bureau for Food Security (USAID) 594100000 Bureau for Global Health (USAID) 1625573000 Bureau for Policy NULL Bureau of Arms Control NULL Bureau of Democracy NULL Bureau of International Narcotics and Law Enforcement Affairs (DOS) 392741000 Bureau of International Security and Nonproliferation (DOS) 467140000 Bureau of Oceans and International Environmental and Scientific Affairs (DOS) 240616000 Bureau of Political-Military Affairs (DOS) 425648000 Bureau of Population NULL Burkina Faso 56367000 Burma 83800000 Burundi 79006000 Cambodia 170147000 Cameroon 27646000 Cape Verde 1257000 Central Africa Regional Office (USAID) 45798000 Central African Republic 10268000 Central America Regional Office (USAID) 61600000 Central Asia Regional Office (USAID) 35832000 Chad 91581000 Chile 2455000 China 27150000 Colombia 843218000 Comoros 252000 Costa Rica 5255000 Cote d'Ivoire 293168000 Croatia 9546000 Cuba 40000000 Cyprus 7000000 Czech Republic 14892000 Democratic Republic of Congo 516246000 Djibouti 15013000 Dominican Republic 64126000 East Africa Regional Office (USAID) 125709000 Ecuador 58891000 Educational and Cultural Affairs (DOS) 5000000 Egypt 3213739000 El Salvador 72263000 Estonia 7262000 Ethiopia 1315017000 Eurasia Regional Office (DOS & USAID) 116915000 Europe Regional Office (DOS & USAID) 37217000 Foreign Assistance Program Evaluation 600000 Gabon 612000 Georgia 236853000 Ghana 378928000 Greece 202000 Guatemala 196109000 Guinea 37257000 Guinea-Bissau 25000 Guyana 24909000 Haiti 757177000 Honduras 133242000 Hungary 3847000 India 250762000 Indonesia 432387000 International Fund for Ireland (USAID) 5000000 International Organizations (DOS) - ICAO International Civil Aviation Organization 1881000 International Organizations (DOS) - IDLO International Development Law Organization 1188000 International Organizations (DOS) - IMO International Maritime Organization 792000 International Organizations (DOS) - International Chemicals and Toxins Programs 7260000 International Organizations (DOS) - International Conservation Programs 15500000 International Organizations (DOS) - International Panel on Climate Change / UN Framework Convention on Climate Change 23500000 International Organizations (DOS) - Montreal Protocol Multilateral Fund 56232000 International Organizations (DOS) - Multilateral Action Initiatives 2000000 International Organizations (DOS) - OAS Development Assistance 8250000 International Organizations (DOS) - OAS Fund for Strengthening Democracy 7440000 International Organizations (DOS) - UN OCHA UN Office for the Coordination of Humanitarian Affairs 5940000 International Organizations (DOS) - UN Voluntary Funds for Technical Cooperation in the Field of Human Rights 2772000 International Organizations (DOS) - UN Women (formerly UNIFEM) 8000000 International Organizations (DOS) - UN Women Trust Fund (formerly UNIFEM Trust Fund) 7500000 International Organizations (DOS) - UN-HABITAT UN Human Settlements Program 3800000 International Organizations (DOS) - UNCDF UN Capital Development Fund 1905000 International Organizations (DOS) - UNDF UN Democracy Fund 9510000 International Organizations (DOS) - UNDP UN Development Program 153535000 International Organizations (DOS) - UNEP UN Environment Program 15400000 International Organizations (DOS) - UNESCO/ICSECA International Contributions for Scientific NULL International Organizations (DOS) - UNFPA UN Population Fund 77700000 International Organizations (DOS) - UNHCHR UN High Commissioner for Human Rights 5000000 International Organizations (DOS) - UNICEF UN Children's Fund 258355000 International Organizations (DOS) - UNVFVT UN Voluntary Fund for Victims of Torture 11700000 International Organizations (DOS) - WMO World Meteorological Organization 4180000 International Organizations (DOS) - WTO Technical Assistance 2290000 International Organizations and Development Institutions (US Treasury) - African Development Bank (AfDB) 32417720 International Organizations and Development Institutions (US Treasury) - African Development Fund (AfDF) 172500000 International Organizations and Development Institutions (US Treasury) - Asian Development Bank (AsDB) 106586000 International Organizations and Development Institutions (US Treasury) - Asian Development Fund (AsDF) 100000000 International Organizations and Development Institutions (US Treasury) - Clean Technology Fund (CTF) 184630000 International Organizations and Development Institutions (US Treasury) - Global Agriculture and Food Security Program (GAFSP) 135000000 International Organizations and Development Institutions (US Treasury) - Global Environment Facility (GEF) 89820000 International Organizations and Development Institutions (US Treasury) - Inter-American Development Bank (IDB and FSO) 75000000 International Organizations and Development Institutions (US Treasury) - Inter-American Investment Corporation (IIC) 4670000 International Organizations and Development Institutions (US Treasury) - International Bank for Reconstruction and Development (IBRD) 117364344 International Organizations and Development Institutions (US Treasury) - International Development Association (IDA) 1325000000 International Organizations and Development Institutions (US Treasury) - International Fund for Agricultural Development (IFAD) 30000000 International Organizations and Development Institutions (US Treasury) - Multilateral Investment Fund (MIF) 25000000 International Organizations and Development Institutions (US Treasury) - Strategic Climate Funds (SCF) 49900000 Iraq 2630487000 Israel 6150000000 Jamaica 18611000 Jordan 1451600000 Kazakhstan 135491000 Kenya 1259768000 Kosovo 134182000 Kyrgyz Republic 120515000 Laos 16904000 Latin America and Caribbean Regional Office (USAID) 90688000 Latvia 7027000 Lebanon 429931000 Lesotho 44550000 Liberia 422469000 Libya 7046000 Lithuania 7525000 Macedonia 38977000 Madagascar 148273000 Malawi 385345000 Malaysia 4779000 Maldives 6633000 Mali 407401000 Malta 300000 Marshall Islands 2096000 Mauritania 18435000 Mauritius 265000 Mexico 747054000 Micronesia 1992000 Middle East Multilaterals (DOS) 1500000 Middle East Partnership Initiative (DOS) 70000000 Middle East Regional Office (USAID) 5000000 Middle East Regional Office Cooperation (DOS) 5000000 Middle East Response Fund 5000000 Moldova 60600000 Mongolia 17675000 Montenegro 12765000 Morocco 84891000 Mozambique 772019000 Multilateral Food Security Programs 14600000 Multinational Force and Observers (DOS) 28000000 Namibia 190595000 Near East Regional Democracy (DOS) 35000000 Near East Regional Office (DOS) 142000000 Nepal 189293000 Nicaragua 43067000 Niger 73954000 Nigeria 1310419000 Office of Development Partners (USAID) 44124000 Office of Innovation and Development Alliances (USAID) 86418000 Office of the Coordinator for Counterterrorism (DOS) 260291000 Office of the Global AIDS Coordinator (DOS) 3747032000 Office to Monitor and Combat Trafficking In Persons (DOS) 39528000 Oman 23788000 Pakistan 3688232000 Panama 10208000 Papua New Guinea 12500000 Paraguay 12633000 Peru 194510000 Philippines 321883000 Poland 66465000 Pooled Funding 73000000 Portugal 125000 Regional Office Development Mission-Asia (USAID) 124480000 Republic of Congo 191000 Romania 28604000 Russia 166485000 Rwanda 438475000 S/GPI - Special Representative for Global Partnerships 3000000 S/SRMC - Special Representative to Muslim Communities 3000000 Samoa 155000 Sao Tome and Principe 298000 Saudi Arabia 9000 Senegal 230702000 Serbia 77887000 Seychelles 235000 Sierra Leone 40375000 Singapore 500000 Slovak Republic 4153000 Slovenia 2319000 Somalia 385059000 South Africa 1104865000 South America Regional Office (USAID) 21530000 South Asia Regional Office (USAID) 6050000 South Sudan 619577000 South and Central Asia Regional Office (DOS) 17048000 Southern Africa Regional Office (USAID) 61200000 Sri Lanka 35599000 Sudan 196024000 Suriname 661000 Swaziland 77256000 Syrian Arab Republic 55500000 Taiwan 500000 Tajikistan 114399000 Tanzania 1054710000 Thailand 26509000 The Bahamas 733000 The Gambia 231000 Timor-Leste 32648000 Togo 1028000 Trans-Sahara Counter-Terrorism Partnership (DOS) 4500000 Trinidad and Tobago 649000 Tunisia 95912000 Turkey 11147000 Turkmenistan 19474000 U.S. Department of Defense - World-Wide 103830000 U.S. Department of the Treasury - Office of Technical Assistance 27000000 USAID Capital Investment Fund 318900000 USAID Development Credit Authority Admin 16600000 USAID Forward: Program Effectiveness Initiatives 71773000 USAID Inspector General Operating Expense 102500000 USAID Operating Expense 2850720000 Uganda 988628000 Ukraine 287225000 Unallocated Earmarks 64922000 Uruguay 1598000 Uzbekistan 56212000 Venezuela 11000000 Vietnam 233481000 West Africa Regional Office (USAID) 168925000 West Bank and Gaza 1023656000 Western Hemisphere Regional Office (DOS) 416700000 Worldwide 26707979 Yemen 255372000 Zambia 713595000 Zimbabwe 245074000 Time taken: 13.766 seconds, Fetched: 250 row(s) hive>