{"question_id":"hellaswag:00007843","item_index":0,"attempt":0,"prompt_hash":"2b00dd2efd796499959dd0bded27883f8b5e19b27db78746dc82485008f2061e","question":"[header] How to cite a book [title] Familiarize yourself with the accepted citation styles. [step] Use only one style when citing a book reference-each style has very specific rules about capitalization, punctuation and placement of the facts. That said, all styles are designed for the same purpose: to give proper credit where credit is due.\n\nA. It is not necessary to include more than your attention span. You could include reading journal articles (if they were used before) or writing your own start point book reviews (if they are not never published by the author).\nB. Listed below are the most common styles : [substeps] Modern language association (mla) style. This is a citation style most often used within the liberal arts and humanities departments of universities.\nC. [substeps] Use a capital letter or a comma after the reference terms of a book reference. This will refer to the same format in a sentence-to-sentence format.\nD. [substeps] There are several great writers' styles for reference: paul golding, glenwhelp, dell, k. paul wilson (1960s innovator), william investments investors, allan cannon (1938), bob n.","prompt":"[header] How to cite a book [title] Familiarize yourself with the accepted citation styles. [step] Use only one style when citing a book reference-each style has very specific rules about capitalization, punctuation and placement of the facts. That said, all styles are designed for the same purpose: to give proper credit where credit is due.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5585,"error":null,"output":"B"}
{"question_id":"hellaswag:00006278","item_index":1,"attempt":0,"prompt_hash":"c6f497c808d72bd987f2dbf8e68f52d0d4ede36d0dafed6488799b49fc90c5c7","question":"[header] How to be emo in middle school [title] Learn what it means to be emo. [step] Before you can become emo, you have to understand exactly what it means. \" emo \" is short for emotive, and people who consider themselves to be emo are usually in touch with their emotions.\n\nA. They don't feel the need to hide how they feel, even when they are sad or angry. [substeps] While music, clothing, hair, and makeup can play a large role in being emo, being comfortable with your feelings and expressing them is the most important part.\nB. This is an important step to emo, so have a good understanding of your gender. You don't have to have what everyone else has.\nC. You don't need a dictionary dictionary and stick with the \" emotive \". Instead, learn what emotive means, like \" i am \", \" i am skinny \", \" i am strong and strong \", \" i'm confident in my body \", etc.\nD. You can't be emo without being emo. [substeps] If you're emo, you want an emo vibe that's unique to you.","prompt":"[header] How to be emo in middle school [title] Learn what it means to be emo. [step] Before you can become emo, you have to understand exactly what it means. \" emo \" is short for emotive, and people who consider themselves to be emo are usually in touch with their emotions.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5583,"error":null,"output":"A"}
{"question_id":"hellaswag:00004307","item_index":2,"attempt":0,"prompt_hash":"0729af895014952a23dca4ba1c3f266ea94c340658c7cadbe54a36c7090f4277","question":"[header] How to make a smoke bomb [title] Gather three or four ping-pong balls. [step] For this method, you don't have to buy any extra ingredients other than basic household supplies, like aluminum foil, a pencil, screwdriver, and scissors. [title] Cut a hole in the top of one of the balls.\n\nA. [step] You can poke a hole with a screwdriver or a knife. [title] Cut the other balls into small pieces and put them in the hole.\nB. [step] Use a knick knack knife to create an x around one of the balls. [title] Punch a hole in the bottom of one of the balls.\nC. [step] Align the hole in the top ball with the one that's down. [substeps] The hole needs to be about six-ten inches deep and about two inches wide.\nD. [step] Grab a piece of foil about 15 inches from the edge of the ball. Get a zig-zag or intricate pattern.","prompt":"[header] How to make a smoke bomb [title] Gather three or four ping-pong balls. [step] For this method, you don't have to buy any extra ingredients other than basic household supplies, like aluminum foil, a pencil, screwdriver, and scissors. [title] Cut a hole in the top of one of the balls.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5582,"error":null,"output":"A"}
{"question_id":"hellaswag:00003431","item_index":3,"attempt":0,"prompt_hash":"727d315e3ed206fc122f7d8197efd788deac0306408e154b804ecdd21450d7f5","question":"[header] How to apply shadow on hooded eyes [title] Prime your eyelids. [step] While this is important for everyone, it's especially critical for those with hooded eyes. Primer creates a base for your makeup that helps it stay put and last all day.\n\nA. [substeps] If you use liquid primer, try it first before you start applying it on your eyes. Otherwise, you'll try a different primer beforehand.\nB. Primer also keeps your eyelids from getting heavy and swollen. To prime your eyelids : [substeps] Apply an eyeshadow to your lids using a dark gray eye liner or gel instead of a cream or gel liner.\nC. Because hooded eyes are prone to smudging and smearing, primer can make all the difference. [substeps] Use your fingertip to apply the primer, and allow it to absorb into your skin for a minute or so before proceeding.\nD. Apply a primer to your eyelids and eyelid over the primer you applied earlier. [substeps] Use your powder brush, primer, or a makeup sponge to carefully prime your eyelids.","prompt":"[header] How to apply shadow on hooded eyes [title] Prime your eyelids. [step] While this is important for everyone, it's especially critical for those with hooded eyes. Primer creates a base for your makeup that helps it stay put and last all day.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5582,"error":null,"output":"C"}
{"question_id":"hellaswag:00004697","item_index":4,"attempt":0,"prompt_hash":"913f8d7c63a6bd3eb841d1b5937145eae25484931ab4eacd853b7e87c34f133f","question":"[header] How to get favorites on your deviantart photo [title] Create your own username. [step] Have a name that can be easily recognized and defines you as a person. [substeps] Your username doesn't have to be your first name, but some well-known deviantart members remember their art style through their da names.\n\nA. Avoid using names of famous celebrities or character from movies, video games, or animes. To change your username, you have to purchase premium membership.\nB. If you don't want people to recognize you as a person, try to keep the name honest and easy to identify. You can do this by including your full name and what year it is, but stay away from androgynous names.\nC. For instance, president christian founder of the white house had a username that \" peter \" (modern), and there are so many other names available today. In the wild, some names are simply variations of the last name.\nD. [title] Create usernames for your favorite characters. [step] Depending on your theme and design, you may have many different options.","prompt":"[header] How to get favorites on your deviantart photo [title] Create your own username. [step] Have a name that can be easily recognized and defines you as a person. [substeps] Your username doesn't have to be your first name, but some well-known deviantart members remember their art style through their da names.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6999,"error":null,"output":"A"}
{"question_id":"hellaswag:00002238","item_index":5,"attempt":0,"prompt_hash":"fec39366eba5407a1ab70d12b602a6e2db1b5067ae72c45352bc83158493c99e","question":"The man lays down and had a hard time breathing. After putting down sand he drinks a warm beverage. we\n\nA. see the man underwater swimming.\nB. see the ending credits screen.\nC. then see two guys swimming nak * d and drinking.\nD. see no other man and the other man is shown shirtless.","prompt":"The man lays down and had a hard time breathing. After putting down sand he drinks a warm beverage. we","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6998,"error":null,"output":"A"}
{"question_id":"hellaswag:00009247","item_index":6,"attempt":0,"prompt_hash":"ecbbc550443eddccdd0fe33a38ff6fb5944fd26bd0d4a2226bc881059ab1cc25","question":"[header] How to decide whether to go abroad for dental treatment [title] Start your search on google or some other search engine. [step] Make a list of the clinics that are featured and that appeal to you. Remember the sponsored clinics have paid to be on the first position, they have not necessarily been selected by google as suiting your criteria.\n\nA. [title] Find out how long the clinic has been offering treatment to overseas patients. [step] If they have been around a long time then they probably have a good track record.\nB. [title] Ask around for \" dentists for dental care \" posters or \" in-clinic treatments \" flyers. [step] Also, make sure you get the name of appropriate dr.\nC. [title] Leave notes at the clinic each day. [step] A written note is always needed in case you are unable to attend the appointment while you are away.\nD. Checking the number of clinics you wish to visit and contacting them makes the decision much easier and easier. [title] Visit a conversation, tea or even facial shop to try to talk to a dentist, he or she will be able to check your dental work and provide you with mouth health options.","prompt":"[header] How to decide whether to go abroad for dental treatment [title] Start your search on google or some other search engine. [step] Make a list of the clinics that are featured and that appeal to you. Remember the sponsored clinics have paid to be on the first position, they have not necessarily been selected by google as suiting your criteria.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6998,"error":null,"output":"A"}
{"question_id":"hellaswag:00002350","item_index":7,"attempt":0,"prompt_hash":"2e760a84b9b4d926179363d75030d949c8fb05bb8f9506c33d700d932fdcef39","question":"People are sitting on tubes in the snow. they\n\nA. stand in a line and start going down a snowy hill.\nB. are going over a large hill.\nC. begin sledding down the hill on the tubes.\nD. stand on the top of the tubes.","prompt":"People are sitting on tubes in the snow. they","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5584,"error":null,"output":"C"}
{"question_id":"hellaswag:00001092","item_index":8,"attempt":0,"prompt_hash":"f7ffb7ee13be9d03e2b2c87bfbae3a0648dc61456559b3fcf02cea7f28b90a4e","question":"Then she take off hr glasses and show how to put the contact on the eye, she grab it with the index finger and put it in the eye. and\n\nA. then put the contact in her eye to work.\nB. finally puts the lens popped with the thumb and rub the glass on the skin.\nC. then she do the same with the other contac in her other eye.\nD. then puts put her finger halfway in to the eye, then close the contact screen.","prompt":"Then she take off hr glasses and show how to put the contact on the eye, she grab it with the index finger and put it in the eye. and","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5581,"error":null,"output":"C"}
{"question_id":"hellaswag:00004433","item_index":9,"attempt":0,"prompt_hash":"d083eb2302e326293cc2374baa4e4b9a6120f7d91e6da6a9d0ff050ad6101012","question":"[header] How to clean pearls [title] Mix a mild dishwashing detergent with water. [step] You can wash pearls in warm, soapy water, but it's important to use a mild detergent. Add 1 teaspoon (5 ml) of soap to 1 quart (946 ml) of warm water to dilute it, and mix well to create a soapy solution.\n\nA. [substeps] You can use any kind of soap to clean pearls, from dish soap to dish detergent to baby shampoo. You can even prepare the mixture specially to wash pearls by hand if you want to avoid unnecessary heating.\nB. Ensure that the water is warm and not hot because it can damage the pearls, especially if they are real. [substeps] A gentle dishwashing liquid or a laundry detergent meant for delicates works well for washing pearls.\nC. [substeps] You should use an all-natural dishwashing detergent if you use a dishwasher. Using a normal dishwasher detergent will ruin the pearls, which can make them hard to clean.\nD. [substeps] Pearls are delicate, so avoid adding too much, which could damage the pearls. [title] Apply a mixture of white vinegar and baking soda.","prompt":"[header] How to clean pearls [title] Mix a mild dishwashing detergent with water. [step] You can wash pearls in warm, soapy water, but it's important to use a mild detergent. Add 1 teaspoon (5 ml) of soap to 1 quart (946 ml) of warm water to dilute it, and mix well to create a soapy solution.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6999,"error":null,"output":"B"}
{"question_id":"hellaswag:00004286","item_index":10,"attempt":0,"prompt_hash":"ee21f70f4a6a4d4ab24ed20a35a707e79bfdef69e1de96eaad9e0511d0654c70","question":"[header] How to manage your beard [title] Eat a healthy diet. [step] A healthy diet generally promotes healthy skin and hair, and this includes your beard. [title] Don't succumb to the itch.\n\nA. [step] Beards often remind people of their beards once they realize they've dislodged their actual beard. You may not realize it, but bits of beard hair will develop into small blackheads to the shape of beard hairs if an adult is not careful.\nB. [step] Itching may persist in new beard growths at first, but it will go away on its own by the end of the day. [title] Wear a beard with a rigid center.\nC. [step] When you're trying to maintain your beard, the itch is the worst thing you can do! It is likely to last even for a few days or weeks before the itch disappears. [title] Write down everything you're thinking about about in your head.\nD. [step] As the hair transitions from stubble to a full-fledged beard, you are likely to encounter several weeks of itch. The itching is unpleasant, but don't despair because it will go away.","prompt":"[header] How to manage your beard [title] Eat a healthy diet. [step] A healthy diet generally promotes healthy skin and hair, and this includes your beard. [title] Don't succumb to the itch.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5581,"error":null,"output":"D"}
{"question_id":"hellaswag:00008296","item_index":11,"attempt":0,"prompt_hash":"707f4f7b252f9e8654fadd92a33f018d222105e9a082a5fc1da13b7a53504b83","question":"[header] How to teach kids how to draw [title] Make art part of the routine. [step] Incorporate art-making into playtime. Make an art-zone if you want to isolate the mess.\n\nA. Fostering your child's sense of focus and excitement can make art fun-something that children will someday be able to do. Art-by doing something one they enjoy, it improves their whole life.\nB. Your child may have troubles learning this art as they remain trapped. Incorporate art onto the routine by having a friend or another child draw a picture together.\nC. Have kids work on side by side, making a space out of the mess for each person to work on. [substeps] If you want to create art-zones, you could also build individual mess zones.\nD. Tape down paper for them to draw on and spill on, and make a smock out of old clothes. Taping paper on a table can help a small child focus on the motion of drawing, without having to hold down and adjust the drawing paper.","prompt":"[header] How to teach kids how to draw [title] Make art part of the routine. [step] Incorporate art-making into playtime. Make an art-zone if you want to isolate the mess.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7000,"error":null,"output":"D"}
{"question_id":"hellaswag:00005695","item_index":12,"attempt":0,"prompt_hash":"7a20779c4b0a8bc9962c18833858157211a70b77d461d7001af336937cf8a3be","question":"[header] How to read like a historian [title] Find some background information about the author. [step] Before you even open the work, search online for the author's history. Find out where they graduated and when.\n\nA. Meet with a respected historian to collect details about him or her. Research the author, and pay close attention to his or her background.\nB. Historical period period descriptions will help you point you in the direction of where to look. [substeps] However, don't assume you can just drop hints.\nC. Look up their other publications and what they focused on. You might even find an article discussing why they chose to write this particular work and why they think it is important.\nD. Find out what they studied in graduate school or what degrees they had in high school. [substeps] If the author has records to back up information, you may want to seek out records to help you back up your argument.","prompt":"[header] How to read like a historian [title] Find some background information about the author. [step] Before you even open the work, search online for the author's history. Find out where they graduated and when.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6999,"error":null,"output":"C"}
{"question_id":"hellaswag:00002753","item_index":13,"attempt":0,"prompt_hash":"ac78d002ae21689d70503bc698f7dd2646d3dfd240d8a7f72278a6914501908d","question":"A mom is sitting at the top of a slide with her little baby. she\n\nA. slides down one at a time, and she tends to make a lot of noise.\nB. is moving backward on the playground, walking away.\nC. pulls out several balls as she slides down.\nD. is contemplating going down for a while.","prompt":"A mom is sitting at the top of a slide with her little baby. she","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":9794,"error":null,"output":"D"}
{"question_id":"hellaswag:00005437","item_index":14,"attempt":0,"prompt_hash":"76a5f03bda32c3a5c68da9ae0d4bcab71d9c6622e111ead6f4d0bd15f0abf403","question":"[header] How to participate in clinical trials [title] Talk to your doctor. [step] If you have a chronic illness or medical condition, your primary treatment provider is your best source for clinical trials that might benefit you. Let your doctor know that you are interested in exploring the possibilities.\n\nA. [substeps] Bring your ob/gyn, a professional, with all of your medical information if you are not sure where to begin. Either ask, \" what would be the best treatment program for me? \" or \" do you know of any facilities near me that might benefit me? \" [title] Educate others.\nB. Make sure that your medical provider knows exactly what sort of information you'll want to explore and will be able to help you find the materials you want to explore. [substeps] Even if you don't get infected with lupus, know that you are still a patient for one year, and you will experience no symptoms over time.\nC. [substeps] If you hear of a trial on your own, bring it up to your doctor and let them know you'd like to participate. They can advise you on whether they think you would benefit from the drug or treatment being studied.\nD. If possible, contact your doctor for more information. [substeps] Keep in mind that clinical trials are usually subsidized by a state or federal government agency and some states will not approve conditions they deemed appropriate for clinical trials.","prompt":"[header] How to participate in clinical trials [title] Talk to your doctor. [step] If you have a chronic illness or medical condition, your primary treatment provider is your best source for clinical trials that might benefit you. Let your doctor know that you are interested in exploring the possibilities.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":9798,"error":null,"output":"C"}
{"question_id":"hellaswag:00007281","item_index":15,"attempt":0,"prompt_hash":"305897d5b7bd29c1246e41c2bfe35ee18197663ebb0319d43bcc308f41bcc283","question":"[header] How to control excessive sweating [title] Switch to a stronger antiperspirant. [step] The first thing to do when attempting to combat excessive sweating is to switch to an antiperspirant with a stronger formula. While prescriptions are required for the strongest of these, there are several clinical strength formulas available over-the-counter from brands like dove and secret.\n\nA. They start by giving you ingredients that are mildly irritating to the sweat glands and gradually get stronger the longer you use them. [substeps] Undiluted lemon or lime juice works well.\nB. There are also very few commercially-produced formulas available as they may not be suitable for daytime use. [substeps] If you opt for a prescription antiperspirant, make sure to check the product label.\nC. [substeps] Powders containing estrogen, which relieve the drying effects of sweat, are commonly used over-the-counter formulas. If someone you love or someone is healthy, or a young person experiencing a health issue, the better moisturizer you choose for them.\nD. [substeps] Be aware of the difference between antiperspirants and deodorants. Antiperspirants will actually clog the sweat glands and prevent excess moisture, while deodorants simply mask odors.","prompt":"[header] How to control excessive sweating [title] Switch to a stronger antiperspirant. [step] The first thing to do when attempting to combat excessive sweating is to switch to an antiperspirant with a stronger formula. While prescriptions are required for the strongest of these, there are several clinical strength formulas available over-the-counter from brands like dove and secret.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7003,"error":null,"output":"D"}
{"question_id":"hellaswag:00004677","item_index":16,"attempt":0,"prompt_hash":"3907a4c9987fd00eec658c9354757391903230d3ced18d8c111c93fb8db6a72c","question":"[header] How to switch tabs in chrome [title] Switch to the next tab. [step] Press + to change to the next tab in the window. This will move you to the tab to the right of your current tab.\n\nA. If you are already on the furthest right tab, this will send you to the one on the far left. This works on windows, mac, chromebook, or linux, but some operating systems have additional options : [substeps] You also have the option to use ctrl + pgdn.\nB. [substeps] Click link/click next to update. [title] Press + to switch tabs in the tab to the left.\nC. You may also need to scroll down to find it. [title] Click on the tab to go back into browsing mode.\nD. Move to the tab to the right of your current tab. [title] Click the \" pin up \" tab.","prompt":"[header] How to switch tabs in chrome [title] Switch to the next tab. [step] Press + to change to the next tab in the window. This will move you to the tab to the right of your current tab.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5587,"error":null,"output":"A"}
{"question_id":"hellaswag:00001131","item_index":17,"attempt":0,"prompt_hash":"86b5022d228ff3e14eb450cd1817ee94e8a97d8a6a69ce1dfbe6770ba79f49de","question":"The action of the lures is shown underwater as several different fish go after the lures. The men are also shown above the ice working their poles and catching several fish. the video\n\nA. continues to show other fish swimming around the sport on the ice.\nB. ends with the closing credits and graphics shown on the screen.\nC. ends with the crowd applauding.\nD. ends with the closing faces of the men and smiling to the camera.","prompt":"The action of the lures is shown underwater as several different fish go after the lures. The men are also shown above the ice working their poles and catching several fish. the video","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5585,"error":null,"output":"B"}
{"question_id":"hellaswag:00000609","item_index":18,"attempt":0,"prompt_hash":"0df9993b1b591468217095793062f654fab4280c1b084fc4046900a9387ae0b2","question":"A man and a woman are in a ring wrestling while a referee is standing in the corner of the ring along with a woman holding a wrestling belt. man and woman\n\nA. are doing wrestling movements in a ring in a dark room and a referee is waching and screaming at them.\nB. are wrestling in punches and kicking in a ring of mats in front of judges.\nC. are holding a wrestling pole as the referee is standing on the other side of the ring.\nD. are standing on opposite ends of the ring holding each other's belts in their hands while lifting each other up in the air upside down.","prompt":"A man and a woman are in a ring wrestling while a referee is standing in the corner of the ring along with a woman holding a wrestling belt. man and woman","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5583,"error":null,"output":"C"}
{"question_id":"hellaswag:00004741","item_index":19,"attempt":0,"prompt_hash":"2ca6a3d5c63f05ac48b8b1c08300749476f08bcebe9c7cbc0f4dffe313d87694","question":"[header] How to fertilize daffodils [title] Fertilize daffodils when the soil needs additional nutrients. [step] Once planted, daffodils usually flower again every year without too much assistance from you. However, because they stay in the ground for a long time, the surrounding soil can become depleted as the plants use up the local nutrients.\n\nA. Most daffodils make excellent compost. [substeps] If the daffodils aren't wintering, each year, fertilize them more slowly with a fertilizer not found in real plants.\nB. [substeps] Daffodils usually bloom in spring, and have multicolored blooms along the stem. During the warmer months, daffodils may produce sugar berries for purchase at local grocery stores.\nC. [substeps] In summer, daffodils can grow anytime between april and october. Spring and summer: daffodils can grow year after season as long as they stay rooted.\nD. Fertilizing is necessary to replace the soil's depleted nutrients. [substeps] Daffodils also reproduce by division of the original bulb, which means more plants eventually occupy the same space.","prompt":"[header] How to fertilize daffodils [title] Fertilize daffodils when the soil needs additional nutrients. [step] Once planted, daffodils usually flower again every year without too much assistance from you. However, because they stay in the ground for a long time, the surrounding soil can become depleted as the plants use up the local nutrients.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5581,"error":null,"output":"D"}
{"question_id":"hellaswag:00007109","item_index":20,"attempt":0,"prompt_hash":"0c36e832e85de1e31fe45f9dead64fa685015bd56da3f90745a027462e62b6ad","question":"[header] How to make zoodle soup [title] Use a spiralizer. [step] This is the preferred tool for many people because it's the fastest and easiest to operate. Unfortunately, it's also the largest and most expensive.\n\nA. The bowels that create zoodles can take a toll on you with long-term use. As a result, zoodles can become extremely hard to work with, especially if they're left in the grinder for too long.\nB. Once you begin to see results with the technique, you may want to start the process over and try another batch. [substeps] Spiralizers can be very expensive, so make sure to check reputable shops before making your own.\nC. To use it, simply wash your zucchini and slice the ends off. Place the zucchini next to the blade and spin.\nD. It's quite discreet, and makes the process much safer for you. [substeps] Spiralizers are small tubes used to filter out milk and water.","prompt":"[header] How to make zoodle soup [title] Use a spiralizer. [step] This is the preferred tool for many people because it's the fastest and easiest to operate. Unfortunately, it's also the largest and most expensive.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5581,"error":null,"output":"C"}
{"question_id":"hellaswag:00000641","item_index":21,"attempt":0,"prompt_hash":"764ffe7850d4ebd73beb8c47cd451bb0e5456e458e2aa8e6322bbdb3d16cff3d","question":"The man hops to his feet and then kneels back down. The man turns to face right and sit son his feet and bounces. the man\n\nA. faces the camera and moves his arms as he speaks.\nB. then stands up and walks out the door while the boys continue to play ping pong while the referee speaks to the team.\nC. does hop scotch while swinging and mainly kneeling.\nD. stands and showcases a mini-fridge.","prompt":"The man hops to his feet and then kneels back down. The man turns to face right and sit son his feet and bounces. the man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6998,"error":null,"output":"A"}
{"question_id":"hellaswag:00000739","item_index":22,"attempt":0,"prompt_hash":"a9f24f5e2f1eb47b891f97f246c8014f172887dd5dff8709eb9a87739dbff747","question":"A man talks in front of a house. A man climbs a ladder and looks at a roof. A man goes through a stack of papers. it\n\nA. is then announcin aluminum on a table.\nB. shows the papers on the screen.\nC. ends with credits and action figures on the screen attached to a fridge.\nD. is then night time again.","prompt":"A man talks in front of a house. A man climbs a ladder and looks at a roof. A man goes through a stack of papers. it","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":9796,"error":null,"output":"B"}
{"question_id":"hellaswag:00006133","item_index":23,"attempt":0,"prompt_hash":"1b0f467393f3671839acc64c3bcdbb636ee5d3125442d8a842f6a5bad7b2ae8c","question":"[header] How to hide the notification bar on android [title] Pull down twice from the top of the screen. [step] This pulls down the notification drawer and then pulls it down further to show the quick settings tiles. [title] Tap and hold wh. Performance. Clearmarks (' image1 _ rendered'); wh. Performance. Mark (' image1 _ rendered'); for several seconds.\n\nA. [step] It will begin caramelizing your notification bar, to enable snagnets, checkpoints, or other options. Try and click the tick next to \" clear notification bar \" before proceeding.\nB. [title] Swipe left once in the options. [step] This will reveal a list of choices; make sure you tap very carefully, as your fingers will slide just from the first selection.\nC. [step] Doing so will re-prepare the notification bar so that it is not visible from the bottom. [title] Lower the notification tray to a discreet, visible position.\nD. [step] It's the gear-shaped icon in the top-right of the notification drawer. After holding on it for a few seconds, the gear icon will spin and roll off the screen.","prompt":"[header] How to hide the notification bar on android [title] Pull down twice from the top of the screen. [step] This pulls down the notification drawer and then pulls it down further to show the quick settings tiles. [title] Tap and hold wh. Performance. Clearmarks (' image1 _ rendered'); wh. Performance. Mark (' image1 _ rendered'); for several seconds.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":9795,"error":null,"output":"D"}
{"question_id":"hellaswag:00005957","item_index":24,"attempt":0,"prompt_hash":"5d7431fdcb75ce7d4f4a8b06edb8909ba633cf31b5e71c2a218a09d418e834e1","question":"[header] How to understand your teenage daughter [title] Really talk to her. [step] Though you can start by asking how her day at school was, don't stop there. It's important for you to really talk with your daughter in order to understand her.\n\nA. You may look for misunderstandings and put her down. But asking your daughter how her day was is crucial to learning how to recognize her strengths and your own weaknesses, which is why we are here.\nB. However, you can still keep basic questions in mind. [substeps] Ask how classes went, what she did friday before school, what she's been doing on her birthday, when her food is, and so on.\nC. [substeps] Begin the conversation with small, open ended questions such as, \" tell me about your day. \" if your daughter doesn't seem to be responding, let her know you're there when she's ready to talk.\nD. At the same time, it is important to test your attraction to her. Ask her questions about her day and see how she reacts to them.","prompt":"[header] How to understand your teenage daughter [title] Really talk to her. [step] Though you can start by asking how her day at school was, don't stop there. It's important for you to really talk with your daughter in order to understand her.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":9795,"error":null,"output":"C"}
{"question_id":"hellaswag:00007917","item_index":25,"attempt":0,"prompt_hash":"3cede662d957eff465827c1070b4cdaabbf780cc7a6e9e65b11345d49973e1e2","question":"[header] How to raise children successfullly [title] Help children learn about nature. [step] Teaching children to love and not fear nature is a vital part of teaching children that this a larger world than their bedroom and games station. It will give them the ability to feel a connection with something other than the house, school and local park.\n\nA. [substeps] The elementary curriculum requires children to learn about nature from birth, and learn how to identify it and understand it. Even at a young age, children become proficient at discovering and appreciating nature and the way it feels to them.\nB. It will also be a learning experience for young children. [substeps] Find new ways to enjoy outdoor sports.\nC. Pack a couple sandwiches, take the kids into the woods and hike a deer trail through the woods. As you go : [substeps] Let the kids ask questions.\nD. Many children like hiking, camping, wildlife photography, or nature watching, so they will enjoy the outdoors with you. [title] Show gratitude for your friends, family, and fellow caregivers.","prompt":"[header] How to raise children successfullly [title] Help children learn about nature. [step] Teaching children to love and not fear nature is a vital part of teaching children that this a larger world than their bedroom and games station. It will give them the ability to feel a connection with something other than the house, school and local park.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6998,"error":null,"output":"D"}
{"question_id":"hellaswag:00007430","item_index":26,"attempt":0,"prompt_hash":"dec067edbb2d3bed93a3b3cdd30078ca9030546f8a962f2fbafd0be8155d2970","question":"[header] How to do your homework on time if you're a procrastinator [title] Organize your notes and files into categories. [step] Procrastinating is easy if your notes are such a mess that you can't find anything you need! Take the time to organize your notes and files. Keep one binder or file folder for each class, and put your notes and assignments in chronological order.\n\nA. This way, if you go into class without your notes, you won't have them when you come back later. [substeps] Get a list of all your work.\nB. [title] Write your assignment due dates in a planner. [step] Go through your class schedule or syllabus and record every due date in a planner.\nC. [substeps] If you are feeling forgetful, place your math, science, and english papers on one of these piles. If you're working hard on a problem, try to avoid focusing on it.\nD. [substeps] If your school provides an organized binder or file folder, use that to organize your papers. If it doesn't, get rid of it.","prompt":"[header] How to do your homework on time if you're a procrastinator [title] Organize your notes and files into categories. [step] Procrastinating is easy if your notes are such a mess that you can't find anything you need! Take the time to organize your notes and files. Keep one binder or file folder for each class, and put your notes and assignments in chronological order.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5581,"error":null,"output":"B"}
{"question_id":"hellaswag:00008694","item_index":27,"attempt":0,"prompt_hash":"f3f93fd4c75e9761e4a4f9cbfb597140765a8f0016f7afae0a9fe5dfb7eee8e9","question":"[header] How to catch a cheating partner [title] Notice if your partner spends less time with you. [step] Your partner may develop sudden new interests or friends if he or she is cheating. He or she may begin working longer hours.\n\nA. He or she may have more commitments such as sports games and nights out. [substeps] Your partner may try to make as big of an impact on your life as possible.\nB. If you ask your partner where he or she is going, your partner may give you a vague answer such as \" out. \" or \" meeting up with some people.\nC. You may notice that your partner spends more time with you when you are not together. Someone who is cheating may spend more time than the other person.\nD. [substeps] Your partner may be dating other girls instead of spending time with you. He or she may not get extra money to spend with you.","prompt":"[header] How to catch a cheating partner [title] Notice if your partner spends less time with you. [step] Your partner may develop sudden new interests or friends if he or she is cheating. He or she may begin working longer hours.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5583,"error":null,"output":"C"}
{"question_id":"hellaswag:00003395","item_index":28,"attempt":0,"prompt_hash":"ac4513fd5234a679cd0ca76e69b2b0e027e79960b127a64dd3bdedc8c069c48a","question":"[header] How to become a special education teacher (usa ) [title] Learn your state and employer requirements. [step] Each state and employer will have their own specific requirements that you will need to meet. Although many states and employers share some requirements in common, there may be some important differences.\n\nA. Some states, such as texas and oregon, don't use special education teachers, but just ensure you meet those requirements. [substeps] For example, the eeoc and the government require special education teachers teach english and russian in addition to high school students in texas.\nB. Take some time and look into your these specific requirements when considering becoming a special education teacher. [substeps] Most states and employers will require you to have earned your bachelor's degree.\nC. [substeps] You will need at least certain certifications in your particular state. Be sure to state what you need to know.\nD. [substeps] Identifying that there are some restrictions might help you decide if teaching jobs apply. For example, you might not be required to have a teaching partner to teach your children.","prompt":"[header] How to become a special education teacher (usa ) [title] Learn your state and employer requirements. [step] Each state and employer will have their own specific requirements that you will need to meet. Although many states and employers share some requirements in common, there may be some important differences.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7000,"error":null,"output":"B"}
{"question_id":"hellaswag:00005639","item_index":29,"attempt":0,"prompt_hash":"3404f846ebc9f81615c5129fba686050f4fb601bdc6d59481821c844c2595aaf","question":"[header] How to teach your kids to be frugal [title] Teach children age-appropriate lessons. [step] All kids want an allowance, but in order to teach your children the true value of money, you may want to make them work for it. You can create a basic weekly allowance depending on good behavior and chore completion at an amount that is comfortable for you.\n\nA. [substeps] A typical two to three month allowance will suffice for a young child. However, if your child asks for more money than they actually need, you may want to adjust this depending on whether they need to work for extra while their parents are away.\nB. Then, create a chart that lists out how much they can earn for doing additional chores, such as washing the car, babysitting younger siblings, or helping with breakfast. [title] Make your kids earn money by completing chores.\nC. Children who want to learn to act frumally by paying their social security checks on time are likely to get them an allowance. It is entirely up to you whether or not you desire to teach your children the value of money.\nD. [substeps] These allowances limit the costs of your house and help cut costs of household supplies and school supplies. Give your kids free raffles and theatre passes at appropriate times.","prompt":"[header] How to teach your kids to be frugal [title] Teach children age-appropriate lessons. [step] All kids want an allowance, but in order to teach your children the true value of money, you may want to make them work for it. You can create a basic weekly allowance depending on good behavior and chore completion at an amount that is comfortable for you.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5585,"error":null,"output":"B"}
{"question_id":"hellaswag:00008288","item_index":30,"attempt":0,"prompt_hash":"4f55d7a28d515ecbb0cb1a8faecdb3b468a6c00cdaac21411a17e0026317fdbb","question":"[header] How to pierce an outie belly button [title] Wash your hands and put on gloves. [step] Use an antibacterial soap to wash your hands. Wash up to your forearms.\n\nA. Pay special attention to your fingers and the area around your fingernails. Once you've washed your hands, put on a fresh pair of latex gloves.\nB. [substeps] Wash the clothes or accessories you were wearing prior to the piercing, such as jewelry. Use a cleanser and liquid dish soap to thoroughly wash your hands.\nC. [title] Use circular motions with your fingers until the skin becomes smooth. [step] Look in the mirror to make sure you've used soap that is approved for outies.\nD. Then rinse your belly button with water and towel dry it. [title] Draw and cut small circles with your pointer fingers.","prompt":"[header] How to pierce an outie belly button [title] Wash your hands and put on gloves. [step] Use an antibacterial soap to wash your hands. Wash up to your forearms.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5586,"error":null,"output":"A"}
{"question_id":"hellaswag:00005471","item_index":31,"attempt":0,"prompt_hash":"2d573848b74b498a32994e415c7bbcac68852445278d4247d0deed56423e9205","question":"[header] How to pronounce oregon [title] Listen to someone say \" oregon \". [step] Visit the state, if possible, or speak with someone who has spent time there. Listen to an audio pronunciation sample : [substeps] Https: // en. Wikipedia. Org/wiki/oregon https: // www.youtube.com/watch? V = wtapsr6knt4 [title] Emphasize the first syllable.\n\nA. [step] Start this question with the sound of \" oregon \" followed by the new word after it. Note if your listener is unsure of the correct first syllable.\nB. [step] Pay attention to how the first syllable is pronounced throughout the word and plant the second syllable in between the words so it sounds like a noun. The easiest and most common way is to say \" oregon.\nC. [step] Make sure you include the first syllable or highest syllable of the word, but don't attach it to the original word. This will indicate to the listener what the original phrase means.\nD. [step] The correct pronunciation uses three syllables: \" or \", \" ih \" and \" gun \". The first syllable is long but the last two syllables are very short.","prompt":"[header] How to pronounce oregon [title] Listen to someone say \" oregon \". [step] Visit the state, if possible, or speak with someone who has spent time there. Listen to an audio pronunciation sample : [substeps] Https: // en. Wikipedia. Org/wiki/oregon https: // www.youtube.com/watch? V = wtapsr6knt4 [title] Emphasize the first syllable.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7001,"error":null,"output":"D"}
{"question_id":"hellaswag:00007455","item_index":32,"attempt":0,"prompt_hash":"a9f996ba715709e251101f8b020a3c38d2c989805ecaae643a6258dcf69c74a0","question":"[header] How to prepare your skin for waxing [title] Let your hair grow. [step] Don't book a waxing appointment until you haven't waxed for least one to two weeks. Ideally, you should have 1/4 inch (. 635 cm) of hair in the area you want to wax.\n\nA. Additionally, waxing will help repair damaged skin and nourish the skin. Let a few days or no more than a couple weeks pass between waxing times to allow for new growth.\nB. Don't wait until your hair has grown beyond this length unless you absolutely have to. Shedding your hairs to cut them will prolong your wait time.\nC. If it's not a waxing appointment, follow the instructions provided in the waxing kit for best results. [title] Apply a 30% moisturizing oil.\nD. Waxing will be more painless and effective if you wait long enough to wax. [substeps] The exception to this is fine hair, like women's facial hair.","prompt":"[header] How to prepare your skin for waxing [title] Let your hair grow. [step] Don't book a waxing appointment until you haven't waxed for least one to two weeks. Ideally, you should have 1/4 inch (. 635 cm) of hair in the area you want to wax.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6545,"error":null,"output":"D"}
{"question_id":"hellaswag:00006486","item_index":33,"attempt":0,"prompt_hash":"36fbd65416660591308969a35b81b5b89ab12d8511be9f058391a1e0b126df98","question":"[header] How to make a bun for short hair [title] Gather your hair into a high ponytail. [step] Use a comb to gather your hair high on your head, either right on top or slightly lower down, according to what you enjoy. Secure your hair with a hair elastic.\n\nA. Make sure it is tightly secured, so your hair won't fall out. [substeps] You might want to use some bobby pins or hair spray to keep your hair in place at the top of your head.\nB. [title] Layer two layers of hair on top of your head. [step] Secure two layers of hair on top of your head, at equal locations-there should be at least 2 layers of hair on each side of your head.\nC. [substeps] If you prefer a retro look, then gather your hair into a high ponytail and twist. If you prefer a more traditional hairstyle, then gather your hair into a low bun (very nice, but not preppy style) and secure with a hair elastic.\nD. [title] Gather your hair into a low ponytail. [step] You may ask the person at the hair salon to split your hair into three sections.","prompt":"[header] How to make a bun for short hair [title] Gather your hair into a high ponytail. [step] Use a comb to gather your hair high on your head, either right on top or slightly lower down, according to what you enjoy. Secure your hair with a hair elastic.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6548,"error":null,"output":"D"}
{"question_id":"hellaswag:00001674","item_index":34,"attempt":0,"prompt_hash":"62edd5cc11639c8ed926f28eb4f4a691f981f6125b0345f7cc246684d59fdcb7","question":"A volunteer named \"irene zimmerman\" speaks to the camera in a field. A volunteer named zeek compton speaks to the camera. they both\n\nA. then demonstrate using the mowing equipment.\nB. use shovels to clear snow off of their cars and trucks.\nC. take turns speaking on camera.\nD. begin rolling a ball over and over using an iron.","prompt":"A volunteer named \"irene zimmerman\" speaks to the camera in a field. A volunteer named zeek compton speaks to the camera. they both","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6551,"error":null,"output":"C"}
{"question_id":"hellaswag:00006683","item_index":35,"attempt":0,"prompt_hash":"94b76d9352edfb9fa55ff93f96692889e03f1fd512a75d6c35d1c5bee0740998","question":"[header] How to style chacos [title] Select leather chacos for a more polished vibe. [step] Choose leather-strapped chacos in neutral shades, such as tan, brown, black, gray, or maroon, for a less casual feel. These will give you a more versatile look than the fabric-webbed chacos, which come off as more athletic.\n\nA. [title] Try fabric chacos for casual events. [step] Choose either solid color or printed chacos for a more informal appearance.\nB. [substeps] Chacos that are made from natural material will last longer. African-pacific chacos are form-fitted, and the laces or buckles won't come off with the addition of fibers.\nC. [substeps] If you prefer a more dressy look, opt for leather-strapped chacos made from reinforced canvas or light but durable leather. They typically come with their own throw pillows and cushions, and are good for swimming or for laying around on your bed.\nD. [title] Keep your chacos clean and hygienic. [step] Chacos should not be stained, processed, or painted with anything dangerous or offensive.","prompt":"[header] How to style chacos [title] Select leather chacos for a more polished vibe. [step] Choose leather-strapped chacos in neutral shades, such as tan, brown, black, gray, or maroon, for a less casual feel. These will give you a more versatile look than the fabric-webbed chacos, which come off as more athletic.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6547,"error":null,"output":"A"}
{"question_id":"hellaswag:00006126","item_index":36,"attempt":0,"prompt_hash":"0b05be2b9eed5eae688b1a23325a5548b769d83d4b5cb39f3c15a14989179a97","question":"[header] How to use everyday groceries in alternative ways [title] Use baking soda as toothpaste. [substeps] Use baking soda twice a month to brush your teeth. It'll help you to get rid of coffee or red wine stains.\n\nA. Eating more than one cup of baking soda for breakfast can help you get rid of as many fake stains as possible. If you don't enjoy brushing your teeth, this is a good time to try using a toothpaste such as bentonite toothpaste or sherbert toothpaste.\nB. Mix two tablespoons of baking soda with a tablespoon of warm water in a bucket until you get a creamy, thick paste. Wet your hands and apply the paste, or put a little baking soda on a cotton pad and rub over your lips, then rinse.\nC. [title] Put one heaping tablespoon of baking soda into a blender. [title] Measure out one tablespoon of some dog-food mix and the juice from three lemons (or you can use packets).\nD. Baking soda can react with acids and bases. This makes baking soda perfect for neutralizing acids produced by bacteria in your mouth.","prompt":"[header] How to use everyday groceries in alternative ways [title] Use baking soda as toothpaste. [substeps] Use baking soda twice a month to brush your teeth. It'll help you to get rid of coffee or red wine stains.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6546,"error":null,"output":"D"}
{"question_id":"hellaswag:00002785","item_index":37,"attempt":0,"prompt_hash":"6557f365ffed84db0b9ae05bba83ccdaaf8648a52cd971040e2d1d916a60bf37","question":"A man is shooting a basketball into a hoop. Another man starts shooting a basketball into the hoop. the first man\n\nA. moves the hoop to the other side of the court.\nB. is laying on the ground holding his chest.\nC. watches the second man.\nD. stands up and throws something onto the court.","prompt":"A man is shooting a basketball into a hoop. Another man starts shooting a basketball into the hoop. the first man","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6549,"error":null,"output":"A"}
{"question_id":"hellaswag:00001715","item_index":38,"attempt":0,"prompt_hash":"a3fe627a2fdb5c6143ac5cc7b63b19fab98fe6b59f815ac209f4828339ba4608","question":"A girl starts brushing her hair. She wraps a curler around her hair and attaches it to her head. she\n\nA. puts another curler on the other side of her head.\nB. pulls it off and spreads it a few times.\nC. then wets her brush and begins braiding her hair.\nD. continues brushing her hair.","prompt":"A girl starts brushing her hair. She wraps a curler around her hair and attaches it to her head. she","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6549,"error":null,"output":"A"}
{"question_id":"hellaswag:00004956","item_index":39,"attempt":0,"prompt_hash":"5b123ff6816f56b4c066f0019b45a74270af0cffb935e8a705b4c895b20ae621","question":"[header] How to help a fearful kitten feel more confident [title] Introduce your kitten to new things slowly. [step] In order to get your cat used to new things, you should introduce the kitten to things slowly. Let your kitten smell a new thing and approach it at their own pace.\n\nA. This may require some practice for a while, but the longer the kitten gets to become familiar with new behaviors, the better your chance of helping your kitten overcome an anxiety attack. [substeps] Handle the kitten for 20 minutes a day.\nB. [substeps] For example, if your kitten seems afraid of pots and pans clanging around in the kitchen, show the kitten a pan and let the cat smell the item. Then show the kitten the clanging sound that the pan makes so it will understand the sound and get used to it over time.\nC. To be sure your kitten loves the interaction, encourage it to interact with new sights, sounds, and smells. You could also try introducing your kitten to new places, such as where you used to take your kitten on trips.\nD. Place your arm around its waist or up near its ears. Pet your kitten and let it smell a new thing.","prompt":"[header] How to help a fearful kitten feel more confident [title] Introduce your kitten to new things slowly. [step] In order to get your cat used to new things, you should introduce the kitten to things slowly. Let your kitten smell a new thing and approach it at their own pace.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6544,"error":null,"output":"B"}
{"question_id":"hellaswag:00005829","item_index":40,"attempt":0,"prompt_hash":"06a3bc8f3d9ff96cab75172a2607cdfa375b549c947318ab4eb6976a07d078e8","question":"[header] How to recognize implantation bleeding [title] Consider the timing. [step] This type of bleeding during pregnancy usually occurs 6 to 12 days after conception--close to the time when the next menstrual cycle is expected. Ask yourself when was the last time you had sex.\n\nA. [title] Determine if you started after being pregnant. [step] In some cases, the result of a feeding may be implantation bleeding; however, do not wait for the numbers to change.\nB. [title] Keep track of how much time or amount of time has passed since you last had sex. [step] Consider whether or not there was a fluctuation in energy during intercourse.\nC. If more than a month or two has passed, then it's unlikely that what you're seeing is implantation bleeding. [substeps] Since implantation bleeding may be confused with the normal menstrual cycle, some women are surprised to discover their pregnancy is a month further along than they originally estimated.\nD. Will the bleeding still be regular after that time? [title] Look for a red or brownish-gray rash. [step] The rash is a reddish-gray ring or a large wound, and it is usually five to six days after release of progesterone (the hormone that controls the menstrual cycle).","prompt":"[header] How to recognize implantation bleeding [title] Consider the timing. [step] This type of bleeding during pregnancy usually occurs 6 to 12 days after conception--close to the time when the next menstrual cycle is expected. Ask yourself when was the last time you had sex.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6547,"error":null,"output":"C"}
{"question_id":"hellaswag:00009185","item_index":41,"attempt":0,"prompt_hash":"b86a5e571ed6f0b4e344a0d5bb0dc8d36b604a584156d7a618bf6724e1532370","question":"[header] How to choose a hvac contractor [title] Ask your friends, family, or neighbors for hvac contractor referrals. [substeps] When you speak to others for referrals, ask about the situation that required them to seek an hvac contractor in the first place. This will give you a better idea regarding whether that specific contractor will best suit your needs.\n\nA. If you can't get direct referrals, search local business directories with customer reviews. Don't base your choice on a nice contractor website only.\nB. [title] Assess the quality of the materials and equipment your contractor is using. [step] Don't let your opinion of the contractor affect the cost of the equipment.\nC. [title] Find local information on where to place an hvac contractor in your area. [substeps] Hvac contractors typically operate in locations with street access to a home or any place with lots of open space.\nD. [title] Review your own services to determine the level of experience you need. [step] It is a good idea to secure advice from a qualified contractor if you need additional experience building hvac filters, buying custom hvac products, and marketing.","prompt":"[header] How to choose a hvac contractor [title] Ask your friends, family, or neighbors for hvac contractor referrals. [substeps] When you speak to others for referrals, ask about the situation that required them to seek an hvac contractor in the first place. This will give you a better idea regarding whether that specific contractor will best suit your needs.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6551,"error":null,"output":"A"}
{"question_id":"hellaswag:00007712","item_index":42,"attempt":0,"prompt_hash":"937be745b376a7e5c0e4186bc4ae4314be7564724a9043a3b2684e97086b0dd8","question":"[header] How to catch eels [title] Select a 6-11 ft (1.8-3.4 m) medium power spinning rod. [step] A shorter rod length will give you greater leverage after hooking the eel, which is especially important since eels are known to be ferocious fighters. A medium power rod will also protect your rod from snapping during the eel fishing process.\n\nA. The straight rod will allow you to swing your rod so your fish can go free. [substeps] Extra length is also a good idea if you're fishing for saltwater fish.\nB. [substeps] Use longer rods if you want to catch longer eels. [title] Pick a medium-capacity spinning reel.\nC. Full strength spinning rods are usually not available, if one is used, and are not suitable for saltwater fishing. If you live where saltwater fish fight, you may try using the powerhead reel with full strength swivel on your blade.\nD. [title] Carefully position the rod as instructed by your manufacturer. [step] You'll be casting the net from short-ranged or mid-ranged lines, with the rod looped over your shoulder.","prompt":"[header] How to catch eels [title] Select a 6-11 ft (1.8-3.4 m) medium power spinning rod. [step] A shorter rod length will give you greater leverage after hooking the eel, which is especially important since eels are known to be ferocious fighters. A medium power rod will also protect your rod from snapping during the eel fishing process.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6547,"error":null,"output":"B"}
{"question_id":"hellaswag:00007702","item_index":43,"attempt":0,"prompt_hash":"074a3c80ccbfd05bdf2786615d5c220016a365e207fe0985a4ac305f43de1134","question":"[header] How to get legal advice [title] Define your legal problem. [step] Before you can start looking for legal information and assistance, you need to be able to explain your problem. A family law issue is going to have a different answer than a problem with a contract or your landlord.\n\nA. If you or someone you know has a medical problem, this should be what you are looking for. [substeps] Medical disorders are similar to relationship issues, so make sure you understand the specifics of those issues before you start your search.\nB. You may also want to make sure your landlord understands where you are coming from or what your problem is. [substeps] Who pays your bills? If it is your landlord, is your \" who pay them \" your problem? Are you paying someone that is not supposed to be paying? [title] Get your landlord's permission.\nC. Make it clear from the outset why a guardianship matter will be the only course of action you take. Ask yourself what you want to do as a result.\nD. [substeps] Have you received legal papers from a court. If so, identify who is suing you and what they want.","prompt":"[header] How to get legal advice [title] Define your legal problem. [step] Before you can start looking for legal information and assistance, you need to be able to explain your problem. A family law issue is going to have a different answer than a problem with a contract or your landlord.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6546,"error":null,"output":"A"}
{"question_id":"hellaswag:00001967","item_index":44,"attempt":0,"prompt_hash":"c9462d21b1eaa360c9e2334c40c0099829dc8b96a4c65cd41fc6fa0d80f0da3a","question":"A baby is sitting in a high chair. an ice cream pop\n\nA. - up nozzles to bring him out of the high chair.\nB. tart is sitting on the counter.\nC. goes into her mouth, and she chews a bite from it.\nD. - tart is put into the woman's mouth.","prompt":"A baby is sitting in a high chair. an ice cream pop","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6549,"error":null,"output":"C"}
{"question_id":"hellaswag:00008587","item_index":45,"attempt":0,"prompt_hash":"e0e91db0af915f7b412dd989a8486edb436881297f5d7e19110abb4f8219113b","question":"[header] How to use teak oil on kitchen cabinets [title] Know what ingredients teak oil products contain. [step] First of all, there is very little oil in any of the commercial grade oil finishes. They are made up of a little oil mixed with mineral spirits and varnish.\n\nA. Poe oil is a little like coconut oil but in a very strong form which gives this oil a very pleasant flavor. Compare oils from all the commercial grade oils and see what each form smells like.\nB. Essential oils should be used sparingly and then applied directly to the cabinet's caulk board with a brush. Second, there are at least a few essential oils that require a little extra oil to get rid of their grease and smudge.\nC. Here's where : [substeps] Whole deglaze is an old oil used for stainless steel, and oils are not waterproof or it becomes lumpy because of the lack of oil in its oil storage. If you want full coverage, opt for a gold or ivory teak oil.\nD. [substeps] The lingo in the cabinetmaking trade is \" wiping varnish \" due to the primary content. You can define it as thinned out varnish with a little teak oil mixed in.","prompt":"[header] How to use teak oil on kitchen cabinets [title] Know what ingredients teak oil products contain. [step] First of all, there is very little oil in any of the commercial grade oil finishes. They are made up of a little oil mixed with mineral spirits and varnish.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6545,"error":null,"output":"B"}
{"question_id":"hellaswag:00009545","item_index":46,"attempt":0,"prompt_hash":"3d0e32bfd1707eb4d1bcd4a5e6de94457be717fc7fb92d36d65d069837a57e3a","question":"[header] How to make a multiplication tables activity board using plastic water bottle caps [title] Plug in hot glue gun, cautiously. [step] Allow the hot glue gun to warm up. [substeps] Don't let your kids use the glue gun themselves.\n\nA. [title] Remove cover cap in step one, as this can quickly melt the cover casing. [step] Clean cap and screw it back into place.\nB. As they get more proficient with the machine with the wires stuck in it, they may well realise that another method is not so helpful. The only way to avoid this is to select a theme board if you do not have one.\nC. Usually, they jump around a bit and bleed, and they think they can solve the problem. If your child is very young, get a few old books and wear them while working on the board.\nD. It's hot, and can cause burns. [title] Use a glue stick to add glue to the construction paper.","prompt":"[header] How to make a multiplication tables activity board using plastic water bottle caps [title] Plug in hot glue gun, cautiously. [step] Allow the hot glue gun to warm up. [substeps] Don't let your kids use the glue gun themselves.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6548,"error":null,"output":"D"}
{"question_id":"hellaswag:00003890","item_index":47,"attempt":0,"prompt_hash":"cff5bf8dbef79d75a6cbdf4c2ed90834007181660ac58840eef7c872ee5d8386","question":"[header] How to start an antique pocket watch collection [title] Attend auctions and estate sales that are selling pocket watches. [step] There are websites online that will give you the location and times for live auctions in your area. Many modern day auctions will also give you a list of items that are being auctioned that day with photos and details.\n\nA. [substeps] In some cases, you may be able to sell your pocket watch on the first day, but in others you may not feel comfortable selling your watch on the first day. To avoid this, travel to all of the major estate sales that you have in the area.\nB. Attending a auction will allow you to see the expected numbers of pieces before purchasing one. [substeps] Some auction websites aren't looking for cheap watches, often requiring individuals to bid for a variety of silver period watches.\nC. Make sure to search for the keyword \" pocket watches \" to find events that you can attend. Your goal early on is not to bid or purchase any watches, but see what buyers are currently looking for and how much they are willing to spend.\nD. Buying pocket watches is a great way to get to know the market first. Just be aware you may not be able to make the auction and estate auction rounds regular.","prompt":"[header] How to start an antique pocket watch collection [title] Attend auctions and estate sales that are selling pocket watches. [step] There are websites online that will give you the location and times for live auctions in your area. Many modern day auctions will also give you a list of items that are being auctioned that day with photos and details.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6548,"error":null,"output":"C"}
{"question_id":"hellaswag:00004153","item_index":48,"attempt":0,"prompt_hash":"d30aeee4df329d3185640b1f044aa98c4a95d3ed0bf6056d0d384496b0fa62e9","question":"[header] How to communicate burial preferences [title] Know your burial options. [step] There are two burial options for you-ground burial and mausoleum burial. With a ground burial, the body is placed in a casket and buried in the ground beneath typically six feet of dirt.\n\nA. In mausoleum burial, your gravestone is placed in the ground. [substeps] If you have an isolated grave with several sides that are aligned, you will bring all of your cemetery spaces together and cover them with rocks and twigs.\nB. [substeps] A \" ground burial \" creates a raised surface for the body to build up on without any additional effort. An \" mausoleum \" provides the longhouse for the body to grow and reduce tissue loss.\nC. A mausoleum burial entails being placed in a casket and then sealed in a wall, sometimes with other caskets. [substeps] In many cities, a mausoleum burial may be required if you live in a city that is below sea level, such as new orleans.\nD. With a mausoleum burial, the body is transferred to the burying ground beneath the ground. [title] Familiarize yourself with public cemetery policy regulations for your cemetery.","prompt":"[header] How to communicate burial preferences [title] Know your burial options. [step] There are two burial options for you-ground burial and mausoleum burial. With a ground burial, the body is placed in a casket and buried in the ground beneath typically six feet of dirt.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6828,"error":null,"output":"C"}
{"question_id":"hellaswag:00009864","item_index":49,"attempt":0,"prompt_hash":"a6a10a58cd89f1365ca99b25775dcb706dd30b706de831bde76006ebc77a1b8e","question":"[header] How to take someone down [title] Block or avoid an attack from your opponent. [step] If someone is coming at you in an attack, you'll want to be prepared to defend yourself. [substeps] Back away from the attacker to put yourself out of reach.\n\nA. If they advance physically against you, knock them over so that they can't escape. Try attacking from a head-level angle in order to hide yourself better.\nB. Place your arms in front of your face to block any punches. Duck under a punch and prepare to counter attack.\nC. If you are trying to pin down the attacker or ensure he cannot hurt you, block the attacker with your non-dominant foot. If you are trying to disarm the attacker, block with your non-dominant foot.\nD. If the attacker does not immediately attack, block their attack with a defensive shield, like a bent knee or a shoulder. Block with both arms locked, preferably going for their head.","prompt":"[header] How to take someone down [title] Block or avoid an attack from your opponent. [step] If someone is coming at you in an attack, you'll want to be prepared to defend yourself. [substeps] Back away from the attacker to put yourself out of reach.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6829,"error":null,"output":"B"}
{"question_id":"hellaswag:00005903","item_index":50,"attempt":0,"prompt_hash":"908c88499c17195258ead9feab408fe2073b527ba6c9d5e1414a392cde17b01a","question":"[header] How to wash lettuce [title] Cut off the lettuce root on leaf lettuce. [step] Use a knife or your hands to break the root off of the lettuce and cut off any wilted spots. Separate the leaves of the lettuce with your hands.\n\nA. Set the wilted lettuce aside while you prepare the last step. [substeps] You can purchase lettuce root at your local grocery store or online.\nB. [substeps] Ensure that all the parts are completely washed and dry. [title] Cut off the leaves to make the dressing dressing dressing.\nC. If the stem breaks off after the cut through the stalk, you won't be able to wash the lettuce right away. [title] Collect or store the wilted leaves and stalk.\nD. [substeps] Be careful when using a knife so you don't hurt yourself. Make sure your fingers are away from the blade when you cut.","prompt":"[header] How to wash lettuce [title] Cut off the lettuce root on leaf lettuce. [step] Use a knife or your hands to break the root off of the lettuce and cut off any wilted spots. Separate the leaves of the lettuce with your hands.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6829,"error":null,"output":"D"}
{"question_id":"hellaswag:00009858","item_index":51,"attempt":0,"prompt_hash":"2ffd000dced36231b18e78dbf796d74503451497f06b7e4de5dcade4b5d9851a","question":"[header] How to function in a multigenerational household [title] Know why this arrangement is in place. [step] Every living arrangement has its benefits and drawbacks, and in order for it to work, the benefits must be recognized and outweigh the drawbacks. When the going gets tough, remembering why you've chosen to live in a multigenerational household will help to cope with any conflicts that may arise.\n\nA. Some of the reasons families choose to live together are : [substeps] Money. Loss of a job or unexpected expenses, like divorce or death in the family.\nB. [substeps] Five quarters of your population can still live in a one-bedroom household, though it's easier to accommodate one less person. [title] Learn what makes a modern lifestyle unique.\nC. [substeps] The primary disadvantages of it are that not sharing a household room is illegal. It might be good for everyone to have a family room as well as a shared living room.\nD. It will also help that you are able to get along with your neighbors, as your personal boundaries will be met as well. [substeps] Your family will not be on your side unless you agree.","prompt":"[header] How to function in a multigenerational household [title] Know why this arrangement is in place. [step] Every living arrangement has its benefits and drawbacks, and in order for it to work, the benefits must be recognized and outweigh the drawbacks. When the going gets tough, remembering why you've chosen to live in a multigenerational household will help to cope with any conflicts that may arise.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6828,"error":null,"output":"A"}
{"question_id":"hellaswag:00000973","item_index":52,"attempt":0,"prompt_hash":"b302a5da047220682f75b16905595c3e692d7e098629f5d66131c48a50fad175","question":"Woman is walking wearing a bikini and eating a chocolate is snowy day. People are in sidewalk watching the girl walks. woman\n\nA. is in a restaurant wearing a bikini and two aiter are watching her.\nB. is high fiving sesame street.\nC. is falling river and is waving to the camera.\nD. do wakediving ieds on a pool and a team and stands in front of the far side of the pool.","prompt":"Woman is walking wearing a bikini and eating a chocolate is snowy day. People are in sidewalk watching the girl walks. woman","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6830,"error":null,"output":"A"}
{"question_id":"hellaswag:00004644","item_index":53,"attempt":0,"prompt_hash":"1f25db18b4b487ef4e9c3222341e5424f3c48045cdb10824cc5f2e1844cd791a","question":"[header] How to treat damaged hair [title] Switch up your shampoo. [step] Look for a moisturizing shampoo, or one for damaged hair. Keep an eye out for shampoos with natural ingredients like jojoba oil.\n\nA. Look for shampoos made with argan oil, which helps prevent hair from drying out and discoloring. [substeps] Moisturizing is also great for fine dry hair, coarse hair, or hair that hasn't reached your desired length.\nB. Look for products made for your hair type. [substeps] For example, cream shampoo gives your hair volume and can help smooth curly hair.\nC. [substeps] Natural ingredients that moisturize will work best to treat your hair. Some chemicals can make your hair feel smoother, but will really cause further damage to the follicle.\nD. Read the label of your shampoo to confirm your brand. Some popular moisturizing shampoos are kale, argan, lemon, and avocado.","prompt":"[header] How to treat damaged hair [title] Switch up your shampoo. [step] Look for a moisturizing shampoo, or one for damaged hair. Keep an eye out for shampoos with natural ingredients like jojoba oil.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6829,"error":null,"output":"C"}
{"question_id":"hellaswag:00000039","item_index":54,"attempt":0,"prompt_hash":"826b936f486dac29af29a5ffc66235c8af9dccae359d643a3930854890c4075f","question":"A person is seen throwing plaster onto a wall while the camera follows the person close behind. the person\n\nA. continues throwing ingredients down and leads into a person kicking a dirty toilet.\nB. gradually throws more and more onto the wall and the camera pans over the finished product.\nC. then uses paper to spread it all over the wall followed by others putting paper through with the plaster.\nD. is now seen putting wax all along the wall while still looking back to the camera.","prompt":"A person is seen throwing plaster onto a wall while the camera follows the person close behind. the person","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6829,"error":null,"output":"B"}
{"question_id":"hellaswag:00001728","item_index":55,"attempt":0,"prompt_hash":"a0f5a2eb8e4bcdd00753d1452f1ad0d6169e7b873a871f2aac1aa40e16debcf5","question":"After, more athletes appear and they follow suit by doing high jumps and clearing the bar without scratching. as the guys\n\nA. in a chase begins, another person who ran into them accidentally hits them and they all bow down to give remorse to the players.\nB. run, they do high jumps off the bar into the water and continue running till their run is over.\nC. continue successfully jumping, they jump on top of the mat and throw their hands in the air because in joy from their success.\nD. continue their routine, next, a large red banner appears upside down and some of them appear sporting long hair.","prompt":"After, more athletes appear and they follow suit by doing high jumps and clearing the bar without scratching. as the guys","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6829,"error":null,"output":"C"}
{"question_id":"hellaswag:00000002","item_index":56,"attempt":0,"prompt_hash":"770b079b5de07442e4a818f4fbebba34c16fc79bbfcfcb1648e68a51021a9ded","question":"Two women in a child are shown in a canoe while a man pulls the canoe while standing in the water, with other individuals visible in the background. the child and a different man\n\nA. are then shown paddling down a river in a boat while a woman talks.\nB. are driving the canoe, they go down the river flowing side to side.\nC. sit in a canoe while the man paddles.\nD. walking go down the rapids, while the man in his helicopter almost falls and goes out of canoehood.","prompt":"Two women in a child are shown in a canoe while a man pulls the canoe while standing in the water, with other individuals visible in the background. the child and a different man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6828,"error":null,"output":"C"}
{"question_id":"hellaswag:00007989","item_index":57,"attempt":0,"prompt_hash":"37ef1f3bf2627416541d023d0fef4320bfc735f8d103050adca266a01d25bb50","question":"[header] How to style a velvet slip dress [title] Layer a long sleeve turtleneck under the dress. [step] Turtlenecks are extremely stylish and will lay flat under the dress without drawing too much attention away from the centerpiece of the outfit. Pick a color that complements the dress, like white, black, dark green, or maroon.\n\nA. You can also mix and match these styles with other garment choices to create an outfit for the evening. [substeps] For a layering effect, add a bulky, asymmetrical v-neck turtleneck that pairs with a petite, tapered neckline.\nB. [substeps] The turtleneck will keep you warm in the winter and play up the artsy vibe of the edgy \" cool girl \" outfit. [title] Put on a denim or leather jacket.\nC. You can even cut off the sleeves if you need to. [substeps] Look for silk or satin velvets or otherwise soft fabrics for added volume.\nD. It should also be equipped with a cufflink so that it doesn't wear through the centre of the dress. [substeps] Grab a shirt and trousers for the winter season, or wear a sweater over the dress in the spring months for warmth.","prompt":"[header] How to style a velvet slip dress [title] Layer a long sleeve turtleneck under the dress. [step] Turtlenecks are extremely stylish and will lay flat under the dress without drawing too much attention away from the centerpiece of the outfit. Pick a color that complements the dress, like white, black, dark green, or maroon.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6829,"error":null,"output":"B"}
{"question_id":"hellaswag:00005416","item_index":58,"attempt":0,"prompt_hash":"4f522ec1affc0e7441eecd200d3742841e018f0aa16bf2710b41c695d9549d2a","question":"[header] How to use body language to keep a guy wanting more [title] Communicate with your eyes. [step] Your eyes are very powerful when it comes to body language. They can indicate interest, and indicate if he's interested as well.\n\nA. When someone's staring at your mouth, smile--looking right into his eyes as they look at yours is a turn-off. If the eye is locked on, hold his gaze for a moment then look away quickly when you feel the urge to stare.\nB. [substeps] One way to identify your eye is to indirectly look at him and hold his gaze. If he looks at you from across the room, then that's as clear a signal as a smile or foot-flicking eye contact.\nC. Eye contact increases your heartbeat and helps to release the hormones associated with love or lust. [substeps] Fix your gaze on a guy you're interested in if you're not already acquainted with him or talking to him.\nD. Look him straight in the eyes when you speak. [substeps] If your eyes are not fixed on him, your pupils will get narrower and don't dilate.","prompt":"[header] How to use body language to keep a guy wanting more [title] Communicate with your eyes. [step] Your eyes are very powerful when it comes to body language. They can indicate interest, and indicate if he's interested as well.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6828,"error":null,"output":"C"}
{"question_id":"hellaswag:00001528","item_index":59,"attempt":0,"prompt_hash":"75fcb71101443c6f847e6f2828a6e743f064cdcdfdd08131afe1293568050705","question":"A man walks along a bare field while a woman walks around beside him. he\n\nA. walks back and fourth preparing to jump and eventually jumps over a pole.\nB. travels around several bases and gets into position in front of a house.\nC. then plays a game of polo with a back and forth.\nD. kicks her shoes out from underneath her while she stands.","prompt":"A man walks along a bare field while a woman walks around beside him. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6748,"error":null,"output":"A"}
{"question_id":"hellaswag:00003438","item_index":60,"attempt":0,"prompt_hash":"de3d2bbf37f7897c446e6214d07bc403dfebb61ede186d7793c4e91d2dda5f6d","question":"[header] How to buy a wig [title] Decide if you prefer a synthetic wig or a human hair wig. [step] Most wigs are either made of out synthetic fibers, real human hair, or a combination of the two. [substeps] Synthetic wigs are especially great for holding volume with short hair styles.\n\nA. Synthetic wigs can work better during the winter when it's cold, but you can always purchase them when you're warm. Wigs made out synthetic fibers are great for keeping shoulder length lengths or long bangs longer.\nB. They are often referred to as \" shake-n-go \" wigs because you can simply shake them out as you pick them up and then put them on. If you are experiencing hair loss due to chemotherapy treatment, most hair grows back within six months of your last treatment.\nC. Synthetic wigs are typically made of the dried/rejuvenated space they cover human hair. Human hair wigs are a wig that is most often used on people with long/curly hair.\nD. Synthetic wigs can be found at most wig stores. Wigs made of out synthetic fibers are easy to keep secure, especially when you're going to the wig store.","prompt":"[header] How to buy a wig [title] Decide if you prefer a synthetic wig or a human hair wig. [step] Most wigs are either made of out synthetic fibers, real human hair, or a combination of the two. [substeps] Synthetic wigs are especially great for holding volume with short hair styles.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6748,"error":null,"output":"B"}
{"question_id":"hellaswag:00005632","item_index":61,"attempt":0,"prompt_hash":"7222d0aaeb487d043d9ddee89f314201da0d6eb3bc079f93c06c9417df21518a","question":"[header] How to organize gardening tools [title] Hang a bike basket on the wall to hold a variety of small items. [step] Bike baskets are designed to be easily hung, so they make an ideal option for hanging from the walls in your garden shed. Use them to store miscellaneous items like gloves, seed packets, pegs, or other small items that might be laying around otherwise.\n\nA. [substeps] Avoid baskets with handles that might break and fall easily. If the basket does hit the wall, add sturdy tape or foil to the rim to better hold it in place.\nB. It may seem a little heavy, but using them will help prevent your tools from tipping over when you have to use them so they don't fall out of reach. [substeps] Cycling helmets are an example of a bicycle basket.\nC. [substeps] Handhang your outdoor shed baskets from the ceiling, wall, and ceiling so they can be easily hung from a tree or hanging tools. [title] Place the basket on a support structure, like a wall, to keep the baskets elevated on top of it.\nD. [title] Hang a rake head upside-down from the wall for a unique hand tool holder. [step] A steel rake head makes a unique holder for your garden gloves, trowels, spades, and other small objects.","prompt":"[header] How to organize gardening tools [title] Hang a bike basket on the wall to hold a variety of small items. [step] Bike baskets are designed to be easily hung, so they make an ideal option for hanging from the walls in your garden shed. Use them to store miscellaneous items like gloves, seed packets, pegs, or other small items that might be laying around otherwise.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6749,"error":null,"output":"D"}
{"question_id":"hellaswag:00001007","item_index":62,"attempt":0,"prompt_hash":"6f6cf1f85f0af9c15b9ce021a76aaa285d55edf05ddab36b679909ce536bfcf6","question":"A woman is standing at a table with a pile of scissors and supplies. she\n\nA. then begins knifing her hair.\nB. demonstrates how to cut cellophane and wrap it around a gift.\nC. proudly shows off her cut nails.\nD. is demonstrating how to play the scissors against her nails.","prompt":"A woman is standing at a table with a pile of scissors and supplies. she","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6747,"error":null,"output":"B"}
{"question_id":"hellaswag:00008252","item_index":63,"attempt":0,"prompt_hash":"174fe92eb235399ae402303709919084877ce2e0f599742e26eb55b6dd4b5416","question":"[header] How to inspect a fireplace chimney [title] Remove the ashes and debris from the fire box. [step] Brush the sides and bottom with a stiff broom or chimney brush. Vacuum out the loosened ashes and if possible remove the andirons.\n\nA. You may want to add a layer of sand or weld dust at this stage. Dry off the contents of the fire box evenly with a damp, paper towel and clean away any remaining dirt, soot or debris.\nB. You can sometimes find the fire box by beneath the fireplace fireplace grate. Be sure you use a ladder to access the top.\nC. [substeps] If you are working with flammable materials, such as wood, furniture, or stones, vacuum them away first to ensure they don't get into the fireplace. If working with furniture or stone, be sure to sweep for particles such as sawdust and soot.\nD. [title] Open the screen and glass doors (if so equipped) as wide as possible, and place a large piece of cardboard or a drop cloth in the firebox. [step] Lay on your back and slide into the box as far as you need to so that you can shine a flashlight up the flue.","prompt":"[header] How to inspect a fireplace chimney [title] Remove the ashes and debris from the fire box. [step] Brush the sides and bottom with a stiff broom or chimney brush. Vacuum out the loosened ashes and if possible remove the andirons.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6748,"error":null,"output":"D"}
{"question_id":"hellaswag:00003818","item_index":64,"attempt":0,"prompt_hash":"8ef4a091d8f584bcb12b4b409eec0ac68026abf847d20afa9fcbebe5791312dc","question":"[header] How to get married in north carolina [title] Find your local county clerk's office. [step] Your will have to go to the county clerk's office to obtain a marriage license. Marriage licenses are granted by each county, so it's easiest if you go to the office for the county you live in.\n\nA. Their office will let you know when you will need to go to get your license. If you live in a rural area, this can be relatively convenient and easiest to get your license, but it may take a little time.\nB. You should get marriage licenses for the county you live in. [substeps] In some counties, you can get a marriage license.\nC. The clerk will let you know if your local license is accepted or rejected in your area. [substeps] If you don't live near your local courthouse, the clerk will post a link to your state's marriage license website (http: // www.officialris.\nD. [substeps] You should be able to look up your specific county court office pretty easily. They should have a section on their website listing the \" marriage license \" services offered at each location.","prompt":"[header] How to get married in north carolina [title] Find your local county clerk's office. [step] Your will have to go to the county clerk's office to obtain a marriage license. Marriage licenses are granted by each county, so it's easiest if you go to the office for the county you live in.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6611,"error":null,"output":"B"}
{"question_id":"hellaswag:00009244","item_index":65,"attempt":0,"prompt_hash":"ed2fb2609a58d768fbdbf9465121f5a02b82c7fa70ab7da915e227e5fd4e4f27","question":"[header] How to teach children prediction skills [title] Emphasize patterns. [step] Pattern recognition is essential to prediction skills. Children in preschool are often developing their ability to recognize patterns.\n\nA. You can help them by pointing out patterns in your everyday life. [substeps] You might use cut-out shapes to teach them patterns.\nB. Tell children you will teach them patterns to make predictions instead of giving them predictions that just happen to them. [substeps] For example, you could instruct them to imagine there are four pipes running through a clock.\nC. However, with older children, it is most important to emphasize patterns. Instead of simply saying, \" the weather is going to rain, \" teach your child patterns.\nD. Even adults with dyslexia will often have patterns in what they see and hear. Even if children develop a pattern, they can still become effective in prediction.","prompt":"[header] How to teach children prediction skills [title] Emphasize patterns. [step] Pattern recognition is essential to prediction skills. Children in preschool are often developing their ability to recognize patterns.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6616,"error":null,"output":"A"}
{"question_id":"hellaswag:00002408","item_index":66,"attempt":0,"prompt_hash":"c56b2c3f0b26f8a8a508ad15156659e65e2b0b1f517b38d6d6d3cbe06b165a01","question":"An animal is being held by a person. a person\n\nA. is opening and cutting animal skin.\nB. is being chopped up on a chopping block.\nC. slides into a tub and drags a pet fish with them.\nD. begins clipping the animals nails with an orange tool.","prompt":"An animal is being held by a person. a person","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6616,"error":null,"output":"D"}
{"question_id":"hellaswag:00005881","item_index":67,"attempt":0,"prompt_hash":"fc383a1c0fa36fd71556c22f2845f473fec0ec2334bad9d0543b781313639059","question":"[header] How to fix nicotine stained fingers [title] Exfoliate your skin. [step] Dampen your skin by soaking your fingers in water for a few minutes, but not long enough for the skin to prune. Use a nail file, pumice stone, body brush, salt scrub, or sugar scrub to sand away some of the nicotine stains on your fingers.\n\nA. Use gentle pressure to rub the grit onto the yellowed area of your fingers just until the stain looks faded. [substeps] Do not rub your finger for more than a few seconds since it can cause irritation.\nB. [substeps] Go to the beauty salon to have your nails done, or your dermatologist to do whatever else you need to do. [title] Do oil-free manicures.\nC. [title] Moisturize your hands and feet. [step] After a prolonged visit to the salon, your hands and feet need to get moisturized.\nD. [substeps] Note that this should only be done in a professional setting, with instruments such as a handheld exfoliator, baby toothbrush, or a kitchen toothbrush. [title] Wash your hands with soap and water.","prompt":"[header] How to fix nicotine stained fingers [title] Exfoliate your skin. [step] Dampen your skin by soaking your fingers in water for a few minutes, but not long enough for the skin to prune. Use a nail file, pumice stone, body brush, salt scrub, or sugar scrub to sand away some of the nicotine stains on your fingers.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6609,"error":null,"output":"C"}
{"question_id":"hellaswag:00002593","item_index":68,"attempt":0,"prompt_hash":"bede18fd0ba22f512cbf65fadcd757faa4fadb807b9ff927ce68d234ac3d1624","question":"People pedal in static bikes in a gym. A man raise his left hand and touches his head. also\n\nA. , a man bend forward holding a sleeve.\nB. , a kneeling man touches his temple and move his head.\nC. , a man pick up several bike handles in his arms making gestures.\nD. , a woman rises the left hand and touches her head.","prompt":"People pedal in static bikes in a gym. A man raise his left hand and touches his head. also","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6613,"error":null,"output":"D"}
{"question_id":"hellaswag:00002761","item_index":69,"attempt":0,"prompt_hash":"9c3107f1c86c71565cbb1e1989537da9b5755a68de30739e4b2c1460c905a49b","question":"A man is see speaking to the camera while sitting on a piece of exercise equipment. he\n\nA. adjusts the settings on the machine and begins using it back and fourth.\nB. sits down and attempts to pull the rope on the machine and begins riding inside.\nC. uses a elliptical trainer and moves his fingers around.\nD. begins sitting on top of the equipment and pushing himself down with his legs in the air.","prompt":"A man is see speaking to the camera while sitting on a piece of exercise equipment. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6614,"error":null,"output":"A"}
{"question_id":"hellaswag:00008975","item_index":70,"attempt":0,"prompt_hash":"3cd132a84da09c45b9302a7be159168d9a94c88e30d7fc312a40ea2b2a74b9d0","question":"[header] How to plan a christmas wedding [title] Look into alternative venues to save money. [step] As many events occur around the holidays, it may be pricier to book a traditional venue like a church. If money is an issue, look into alternative venues.\n\nA. [substeps] If you're having a winter wedding in a warmer region, consider marrying in an outdoor park. Usually, the only fee required is a small registration fee.\nB. Either stay at a rural or close-knit venue to save money, or book the venue where you usually do your fancy wedding activities. Before booking your venue, talk to local and national leaders to find out what options are available.\nC. You can either opt to have a renaissance wedding in a park where everyone will be happy to go for an afternoon, or a garden with a wide array of flowers. [substeps] Another option is to have a simple church wedding where people in the church and the cathedral go out shopping for food and edibles.\nD. [substeps] Attending a church service, even if it's not a traditional christmas wedding, can save you money. Also, it may be cheaper because you'll make a special \" guest of honor \" as well.","prompt":"[header] How to plan a christmas wedding [title] Look into alternative venues to save money. [step] As many events occur around the holidays, it may be pricier to book a traditional venue like a church. If money is an issue, look into alternative venues.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6612,"error":null,"output":"A"}
{"question_id":"hellaswag:00009867","item_index":71,"attempt":0,"prompt_hash":"77b70ae5dcba293ac7b12c25ead5194e2f7bc46600c9277b58eeae7c9bb57583","question":"[header] How to use a front kick for self defense [title] Know your limits. [step] Unless you are trained, you should leave your jackie chan moves in the gym. The type of front kick most people imagine (flying at the attacker's torso) is more likely to bring disaster on yourself than your attacker.\n\nA. Plus, your fists just have to make contact with the attacker's face and attack head-on. [substeps] If you're short on space to escape, just go for the head-on.\nB. Practice different kicks like the one jackie chan kick on-to-protect yourself from oncoming attacks. [title] Make sure you allow others to control you.\nC. See things from jackie chan's perspective. [title] Position your legs so that they are over your toes.\nD. The average person does not have the kick speed to pull this off. Put in some time practicing and get a sense of your capabilities, asking a martial artist for advice if possible.","prompt":"[header] How to use a front kick for self defense [title] Know your limits. [step] Unless you are trained, you should leave your jackie chan moves in the gym. The type of front kick most people imagine (flying at the attacker's torso) is more likely to bring disaster on yourself than your attacker.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6610,"error":null,"output":"D"}
{"question_id":"hellaswag:00006802","item_index":72,"attempt":0,"prompt_hash":"863bc21d07a83b006f3a64a21c7ac3c562aec25afae0a6b243792d00dbd7671b","question":"[header] How to heal chapped lips naturally [title] Exfoliate your lips with a sugar scrub. [step] To gently remove flaky skin from your lips and prime them for moisturizer, make a scrub out of a small amount of sugar mixed with olive or coconut oil. Apply the mixture to your lips with gentle pressure from your fingers and leave it on for a few minutes.\n\nA. Then, gently scrub and rinse your lips. Follow this regimen with an application of a natural moisturizer like honey or olive oil.\nB. Wash your lips with your usual exfoliating cleanser. Rinse your lips with cold water and shake them a couple of times to dry them out a bit.\nC. Then, run the sugar scrub over your lips and leave it on for a few minutes more. By using a sugar scrub, you can prevent dryness and blemishes by removing dead skin cells.\nD. Then, rinse it away with lukewarm water. [substeps] Exfoliating will be particularly important if you have oily lips because it can help provide very thin coverage for your lips.","prompt":"[header] How to heal chapped lips naturally [title] Exfoliate your lips with a sugar scrub. [step] To gently remove flaky skin from your lips and prime them for moisturizer, make a scrub out of a small amount of sugar mixed with olive or coconut oil. Apply the mixture to your lips with gentle pressure from your fingers and leave it on for a few minutes.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6609,"error":null,"output":"A"}
{"question_id":"hellaswag:00005689","item_index":73,"attempt":0,"prompt_hash":"75bf5672a9618ce32ee8610179d5ff10ab106b7d834f0d7f590ab8ba76b4cb9e","question":"[header] How to make a cat halloween costume using everyday items [title] Get a headband that is the colour of cat you want to be. [step] If you want to be a black cat, get a black headband. If you want to be an orange cat, get an orange headband.\n\nA. If you want to be an animal, get a wild cat headband. [title] Give the headband a colour that you like.\nB. If you want a color two colour to one then get a one colour headband. [title] Lay down the headband on a sheet of newspaper, a table, or any other flat surface.\nC. All you need to do is make a cat head headband. [substeps] For example, you can get a headband with seven stars by pricking a star with an orange coin or orange marzipan.\nD. [substeps] If you can't find a headband the colour of cat you want to be, get some felt in that colour and wrap it around that headband. [title] Have some felt that is the same colour as the headband and some pink felt for the inside of the ears.","prompt":"[header] How to make a cat halloween costume using everyday items [title] Get a headband that is the colour of cat you want to be. [step] If you want to be a black cat, get a black headband. If you want to be an orange cat, get an orange headband.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6607,"error":null,"output":"D"}
{"question_id":"hellaswag:00003155","item_index":74,"attempt":0,"prompt_hash":"867e210b47010f7d03ed6aac671c3d4d762099826f3fac54a8968efae5d77d2a","question":"A small puppy is rubbing his face in someone's lap. he\n\nA. chews and licks his paw as he cuddles.\nB. looks up, and the person starts shaving the top of his ears with a clipper.\nC. lifts up his face and smiles, looking on the camera.\nD. looks up and smiles and then a woman is giving a news segment about him.","prompt":"A small puppy is rubbing his face in someone's lap. he","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6610,"error":null,"output":"C"}
{"question_id":"hellaswag:00004436","item_index":75,"attempt":0,"prompt_hash":"10bc60a0b5e9a058a437231f37da6e850e77bf4b3d53c511d7a3e596bbb898be","question":"[header] How to teach a blind or visually impaired child to tie their shoes [title] Start giving lessons between ages five and seven. [step] Teachers of the blind and visually impaired often stress the importance of learning age-appropriate skills, which contributes to developing motor functioning and increasing independence. Most children learn to tie their shoes between the ages of five and seven, so a blind child should also start learning during this age range.\n\nA. [substeps] By age five, the child should be able to tie their shoes easily and effortlessly. They do not need to grab the laces or add more pressure to them after they've finished tying the shoes.\nB. [title] Know that having young children learn to tie their shoes may take a considerable amount of time. [step] It may involve more resorting to traditional 1-step training for children who do not have the ability to walk.\nC. [substeps] Start off slow with the child, since learning this skill won't happen overnight. You don't have to start until you feel they're ready.\nD. [title] Gain a mastery of hand-eye coordination. [step] Dementia is often associated with memory loss, which can cause deep nerve damage.","prompt":"[header] How to teach a blind or visually impaired child to tie their shoes [title] Start giving lessons between ages five and seven. [step] Teachers of the blind and visually impaired often stress the importance of learning age-appropriate skills, which contributes to developing motor functioning and increasing independence. Most children learn to tie their shoes between the ages of five and seven, so a blind child should also start learning during this age range.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6609,"error":null,"output":"A"}
{"question_id":"hellaswag:00004107","item_index":76,"attempt":0,"prompt_hash":"84a458b26437f458038a5318fdd3694c9ba6f20f5eeb1c8a0302edda8a5f436b","question":"[header] How to overcome family pressure to be in a religion [title] Understand the reasoning for this article. [step] Perhaps you live in an area where there are few people who think exactly like you do. Somehow, make a connection with another who feels or thinks very similarly and is very accepting to you as a person.\n\nA. [substeps] Talking about how we feel about religion can help draw this person into the will of god. While listening to you talk about what you think of religion can help you develop a range of reasons for believing, we now have insights to put into view what we see and think of today.\nB. If this is the case, you can do everything you can to stop the romantic pressure. If people are negative of you for not being in a religious community, like your church, show them that you are willing to honor the faith of others.\nC. For example, you might have family-brothers, sisters, etc. - who view abortion as something to adore.\nD. This will bolster your confidence in speaking up about your need to be accepted and respected by others. Once you realize that you have a friend or family member with similar beliefs, you should feel the strength to discuss openly your need for further social acceptance.","prompt":"[header] How to overcome family pressure to be in a religion [title] Understand the reasoning for this article. [step] Perhaps you live in an area where there are few people who think exactly like you do. Somehow, make a connection with another who feels or thinks very similarly and is very accepting to you as a person.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6611,"error":null,"output":"D"}
{"question_id":"hellaswag:00000481","item_index":77,"attempt":0,"prompt_hash":"144568ced8206e31ba4433a1b02c7e19ae134f4353155e812272727908f43b36","question":"A person is standing in the heavy falling snow. they\n\nA. are trying to clear the snow off the sidewalk.\nB. flip over and fall sitting in the snow.\nC. play a game of skiing.\nD. are unable to get up because of the large weight on their backs.","prompt":"A person is standing in the heavy falling snow. they","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6609,"error":null,"output":"A"}
{"question_id":"hellaswag:00009833","item_index":78,"attempt":0,"prompt_hash":"d5195d166e89f4fcdd8ebf3aadd3c7776f2cf264899cf6e4743baceee1f2bc5e","question":"[header] How to refill a stapler [title] Open up the standard or mini stapler. [step] On most staplers, you lift the top. There may be some latches on the side that need to slide to unlock it.\n\nA. In these cases, you'll need to use a screwdriver. [substeps] Keeping that in mind, if you have your settings on \" off, \" there should be a picture of a click-and-release cord somewhere on the handle.\nB. Consult a handbook or refer to a menu for the type of stapler you will need to use to refill your pencils. [substeps] The card stock is found beneath the teeth on most basic printer models, including those 97.3-10 and 36.5-10.\nC. In some cases, the stapler doesn't open upward; instead, a large push button on the top rear ejects the tray out the front. [substeps] Examine the stapler you're refilling to work out which method will open the stapler tray.\nD. When the top opens, you will also have to unscrew some screws that are holding it in place. Some standard staplers may still not have the release key in order to unscrew these.","prompt":"[header] How to refill a stapler [title] Open up the standard or mini stapler. [step] On most staplers, you lift the top. There may be some latches on the side that need to slide to unlock it.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6610,"error":null,"output":"D"}
{"question_id":"hellaswag:00007820","item_index":79,"attempt":0,"prompt_hash":"31941ffffbacbae8591943657bf52802fda671b8b87c1eb1697b73f982bed315","question":"[header] How to tell if your face is well suited to bangs [title] Go for side-swept bangs if you have a round face. [step] These bangs are styled like the name would imply-they flow over the forehead to one side of your face. The best types of side-swept bangs for round faces are thick and full, rather than thin and wispy.\n\nA. [title] Have your bangs wavy for fuller ends. [step] If you have curly hair, try brushing it to one side and then having your bangs turned-side-up in a curled ponytail.\nB. These will add a bit of shape and dimension to a round face. Figure out which side of your face you would like your bangs to go and try it out.\nC. [substeps] For this look, you might want to keep your bangs at a slight above your ears, and use a slightly thicker option if you have a round face. [title] Look for short bangs if you have golden undertones.\nD. [substeps] The most noticeable part of the side-swept bangs is the left side that overhangs your forehead. To determine the length of your bangs, move your hair to either side of you.","prompt":"[header] How to tell if your face is well suited to bangs [title] Go for side-swept bangs if you have a round face. [step] These bangs are styled like the name would imply-they flow over the forehead to one side of your face. The best types of side-swept bangs for round faces are thick and full, rather than thin and wispy.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6611,"error":null,"output":"B"}
{"question_id":"hellaswag:00003468","item_index":80,"attempt":0,"prompt_hash":"cebf66a442d8148756ca729910c921de7682b80d6337475df7214f1f8720c181","question":"[header] How to preserve air jordan sneakers [title] Put them in zip locks with few packs of silica. [title] Buy force guards. [step] If you have a ton of shoes but preserving them is worth it as it makes creasing nearly impossible.\n\nA. [title] Buy a disposable metal self cleaning item ready as rocks for pulling or sharpening. [step] Depending on your size and the size of your shoes you need to clean them daily.\nB. If you can't, use a pack in large socks. Duct tape the small spaces or buy new ones with snap snaps for ease of use.\nC. [title] Try not to wear them in bad weather. [step] Even the lightest rain can damage your shoes and cut off a year in its life.\nD. [title] Enlist the help of a sidewalk sweeper. [step] You get the hassle of attempting to stop someone accidentally when they're not even there.","prompt":"[header] How to preserve air jordan sneakers [title] Put them in zip locks with few packs of silica. [title] Buy force guards. [step] If you have a ton of shoes but preserving them is worth it as it makes creasing nearly impossible.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6455,"error":null,"output":"C"}
{"question_id":"hellaswag:00002490","item_index":81,"attempt":0,"prompt_hash":"a544ada4863e0076d0022735ee9adb50d509a9d96c88728026c4e9027859cdcd","question":". a young female dressed in a white jersey and purple shorts\n\nA. is sitting on the courtyard floor holding a violin and smiling to the camera, talking.\nB. is putting on her weightlifting gear and is preparing to begin barbiedoll lifting.\nC. is standing on the edge of a pool stretching.\nD. is standing outside wearing a karate uniform trying to do her basic karate moves in a large open gym that has many wooden walls.","prompt":". a young female dressed in a white jersey and purple shorts","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6457,"error":null,"output":"C"}
{"question_id":"hellaswag:00007550","item_index":82,"attempt":0,"prompt_hash":"112fb16c18953afd19ad7ac2b473e9cfb09bc03a189fe923678e1aff82aea7a6","question":"[header] How to store raspberries [title] Choose a storage container that provides air circulation. [step] You should not store berries in an airtight container. To stay fresh, berries need some air circulation.\n\nA. For the best results, leave the produce and juices fresh, rather than broken or contaminated. [substeps] If you store fruits and vegetables frozen in ice water, freeze the fruits and vegetables for 15 to 20 minutes before storage to provide air circulation and prevent decay.\nB. Too much air circulation can cause the berries to spoil. [substeps] Use a container with at least a couple inches of air circulation--less air gets into the container, resulting in dead, powdery cells.\nC. There are several types of storage containers that should work well for raspberries : [substeps] A tube container (1x8 inch). This type of container will allow to hold more raspberries than plastic or glass tubes.\nD. Use the original container they came in, which usually has slots and holes. If don't have the original container, store them in something like a colander.","prompt":"[header] How to store raspberries [title] Choose a storage container that provides air circulation. [step] You should not store berries in an airtight container. To stay fresh, berries need some air circulation.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6455,"error":null,"output":"D"}
{"question_id":"hellaswag:00004257","item_index":83,"attempt":0,"prompt_hash":"e939ad6de86393f0683f77f472cdd9ce9fb8609918da3d3aed633eef50296799","question":"[header] How to straighten men's hair [title] Wash and condition your hair. [step] Put the blow dryer on the medium heat setting and move it over your hair as you comb it. Use a nozzle attachment to help control the airflow.\n\nA. [substeps] This method pulls out excess heat from your hair while warm. Do not underwash your hair when it is heated, and especially never use blow drying methods such as curling irons and straighteners.\nB. Comb down from the roots to the tips. As the hair dries, it should be straighter than it normally is.\nC. Focus on the roots and consider tying it into a high ponytail. Wrap a towel around your head and head over to the sink.\nD. [substeps] After finishing the blow dryer, give your hair a more thorough wash with a conditioner. This will keep your hair looking neat, hopefully without making it frizzy.","prompt":"[header] How to straighten men's hair [title] Wash and condition your hair. [step] Put the blow dryer on the medium heat setting and move it over your hair as you comb it. Use a nozzle attachment to help control the airflow.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6457,"error":null,"output":"B"}
{"question_id":"hellaswag:00002832","item_index":84,"attempt":0,"prompt_hash":"61262be92db710c07921dbe31a803abbcf9517fd7bfb3e302f766cf4e7fa7ea9","question":"Video footage is then shown of her running alongside a large group of people while many watch on the sides. more pictures\n\nA. are shown of her running and others interviewing her.\nB. are shown of her running while smiling and talking when performing aerobic stretches.\nC. are shown of her finishing the race as well as others.\nD. are shown of her running with others at various speeds.","prompt":"Video footage is then shown of her running alongside a large group of people while many watch on the sides. more pictures","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6455,"error":null,"output":"C"}
{"question_id":"hellaswag:00007058","item_index":85,"attempt":0,"prompt_hash":"3c5887549799255a9a89f270e8e2af9b5b06169edb8780ebf05b4f4828c5e7bc","question":"[header] How to barrel race [title] Pick the right horse. [step] Your horse is the most important piece of your barrel racing team--other than you, of course. You and your horse should be equal partners and work together to learn to succeed at barrel racing.\n\nA. You should : [substeps] Be free to throw flies over your horse's head. These well-timed throws can stun your horse, making him vulnerable to sharp bolts.\nB. In order to fail by running out of time, it is important to appear on the field on time, and listen to your horse as it races. You can use your horse in your attempt to permanently trigger it's own spotting.\nC. If you don't have confidence in your horse, stock up on kits and/or buttock leathers. [substeps] Additionally, winter-related races are great places to focus on the basics of barrel racing.\nD. Some horses are more capable of becoming barrel horses, so you'll want to choose your partner carefully. [substeps] The horse's health is the most important consideration.","prompt":"[header] How to barrel race [title] Pick the right horse. [step] Your horse is the most important piece of your barrel racing team--other than you, of course. You and your horse should be equal partners and work together to learn to succeed at barrel racing.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6454,"error":null,"output":"D"}
{"question_id":"hellaswag:00009690","item_index":86,"attempt":0,"prompt_hash":"d490dbd2ebc15fbec39af0f80054a172cc8d6ee0c68a03f26a5e61155058bd0b","question":"[header] How to get a tebori tattoo [title] Search online for tebori tattoo artists. [step] The internet will be your best resource for finding and reaching out to tebori parlors. Search for tebori-specific tattoo artists.\n\nA. Some tattoo artists only specialize in tattoo businesses, such as the grass tattoo studio, ufc brand, or goodness. Consider websites like shearer and serious studios.\nB. For example, a tattoo parlor called honey tattoo in the ikebukuro district of tokyo provides tebori tattoos. [substeps] Tebori tattoo parlors that are more traditional or located farther from urban centers (like tokyo) may be unwilling to respond to inquiries made in english.\nC. Some parlors do their tattoo curing for free, while others charge a fee for the process. Make sure the artists are licensed, credentials are kept, and mentioned on their websites.\nD. Do some research and submit what looks good at first out of sight. [title] Be prepared to pay a relatively small fee to purchase an inked tattoo.","prompt":"[header] How to get a tebori tattoo [title] Search online for tebori tattoo artists. [step] The internet will be your best resource for finding and reaching out to tebori parlors. Search for tebori-specific tattoo artists.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6454,"error":null,"output":"B"}
{"question_id":"hellaswag:00000246","item_index":87,"attempt":0,"prompt_hash":"2279c201b4da26434706899ce67527e6857025f1c4142d21ccf7933d44b90e6a","question":"A woman walks over to a flower bed and wheelbarrow as she talks. she\n\nA. picks up several bags of flowers on top of the flower bed.\nB. demonstrates how to mulch, picking up the material and showing how to pack it around plants.\nC. then shovels the dirt around the trunk of a large tree.\nD. places the wheelbarrow and supplies inside.","prompt":"A woman walks over to a flower bed and wheelbarrow as she talks. she","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6454,"error":null,"output":"A"}
{"question_id":"hellaswag:00007006","item_index":88,"attempt":0,"prompt_hash":"9dcb0bbe8e55833043c624b62bbe44f51cd7f71319aed4e8817501be4470a0de","question":"[header] How to shave your neck when growing a beard [title] Visualize a line from one ear to the other. [step] Imagine a line that loops from the back edge of each ear down and around the top of your neck. Envision the lines meeting in the center of your neck (below your jaw).\n\nA. [title] Close your eyes and draw a line halfway along the back, below your cheekbones. [step] Be aware that this should be both long and tapered, to simulate a straight neck in this image.\nB. [title] Visualize two lines extending down from your sideburns. [step] Find the rear edges of your sideburns.\nC. [substeps] If you are shaving inside or outside of your ear, concentrate on the lines. Don't be surprised if the lines are poorly defined at first; over time, they'll become more defined as you become more proficient at shaving.\nD. If you feel the top side of your neck rubbing against this line, you probably need to shave for your beard. [substeps] In your dreams, take the entire scene in your mind and try to create a picture that feels real and realistic.","prompt":"[header] How to shave your neck when growing a beard [title] Visualize a line from one ear to the other. [step] Imagine a line that loops from the back edge of each ear down and around the top of your neck. Envision the lines meeting in the center of your neck (below your jaw).","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6454,"error":null,"output":"B"}
{"question_id":"hellaswag:00000545","item_index":89,"attempt":0,"prompt_hash":"c8878ab64055ba3c45f1b752b1cec3b9711513009b7665a5a123decb0c5b640b","question":"Two men and two women are playing table soccer and showing the table. A man and a woman are in a living room playing table soccer. people\n\nA. are in a comercial showing the socer table and playing.\nB. standing around her and in the corner practicing table soccer.\nC. are standing in street behind them watching.\nD. are walking by the table wearing white bumper cars.","prompt":"Two men and two women are playing table soccer and showing the table. A man and a woman are in a living room playing table soccer. people","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6454,"error":null,"output":"C"}
{"question_id":"hellaswag:00001760","item_index":90,"attempt":0,"prompt_hash":"73b0e296bb5028fc5c6f29a0dd58d7483d90065a64ca53abd26f332cc4b88339","question":"A man cranks his lawn on. the man\n\nA. changes his yard and puts floor tiles on the floor.\nB. starts to mow his lawn.\nC. 's lawn is shown in one spot.\nD. rakes in the dead leaves along the ground.","prompt":"A man cranks his lawn on. the man","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6453,"error":null,"output":"B"}
{"question_id":"hellaswag:00006433","item_index":91,"attempt":0,"prompt_hash":"c0556c2b04c2e66ec7a3a10ac3f1769e2e00b3f314568c702f98ad20a0be214b","question":"[header] How to dress for a rehearsal dinner [title] Check the invitation for dress details. [step] The bride and groom or the groom's parents will have sent you an invitation noting the details of the rehearsal dinner. In addition to the time and date of the event, some invitations note the style of dress the hosts wish for guests to wear.\n\nA. Here are some of the different styles of dress you may see listed on a rehearsal dinner invitation : [substeps] Black tie. This style is more common for the actual wedding day, but you may see a request for black tie attire on a rehearsal dinner invitation.\nB. [substeps] For example, you could check an invitation asking guests to wear elegant suits or dresses with empire necklines. Sometimes, a tuxedo provides a casual look without the emphasis of formal attire.\nC. These invitations will indicate their attire so you can tailor your attire accordingly. [title] Get the right attire to attend the rehearsal dinner.\nD. In general, most hosts will want to show you these details. From this information, you should consider deciding what type of attire you would like to wear for a rehearsal dinner.","prompt":"[header] How to dress for a rehearsal dinner [title] Check the invitation for dress details. [step] The bride and groom or the groom's parents will have sent you an invitation noting the details of the rehearsal dinner. In addition to the time and date of the event, some invitations note the style of dress the hosts wish for guests to wear.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6357,"error":null,"output":"A"}
{"question_id":"hellaswag:00000964","item_index":92,"attempt":0,"prompt_hash":"c6265363ac36e85a3edc3d8dcf3b69aa2cb3b1c07f12b9a5192128c930abff1b","question":"A man is working out on an elliptical machine. it\n\nA. is shown in two pieces, each one attached by a person.\nB. seems to be a popular one as he works out.\nC. grabs and guides the device.\nD. shows the screen on the machine.","prompt":"A man is working out on an elliptical machine. it","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6357,"error":null,"output":"D"}
{"question_id":"hellaswag:00000314","item_index":93,"attempt":0,"prompt_hash":"632fb32ac168d6a3121ba49c326ad7ba4d079bdf9bc36bfccc3d13cc7b9467ae","question":"A woman sits in front of a bathroom mirror squishing water in her mouth, as if rinsing her mouth. the woman\n\nA. then spits the water into the sink and looks at her teeth very close in the mirror before the scene fades out.\nB. then opens a hand mirror and tubs a wet cloth, which she pours into her mouth and animates.\nC. moves her fingers towards her mouth and stops.\nD. finishes and reaches for her mouth in the sink.","prompt":"A woman sits in front of a bathroom mirror squishing water in her mouth, as if rinsing her mouth. the woman","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6357,"error":null,"output":"A"}
{"question_id":"hellaswag:00000166","item_index":94,"attempt":0,"prompt_hash":"bd4c91383d1527db882bf0c4b46cfe4178e04dcd568f9d82a49c9c3b9564b7fa","question":"A woman is seen putting makeup on her face and leads into several pictures of makeup. Many more pictures of products are shown with text across the screen. more pictures of products\n\nA. are shown on the screen leads into a woman putting makeup on herself.\nB. are shown and ends with the camera shining off into the distance and the woman walking away.\nC. are shown and ends with a woman hula hooping with herself.\nD. are shown that end with more text and a woman pushing away makeup.","prompt":"A woman is seen putting makeup on her face and leads into several pictures of makeup. Many more pictures of products are shown with text across the screen. more pictures of products","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6357,"error":null,"output":"A"}
{"question_id":"hellaswag:00007409","item_index":95,"attempt":0,"prompt_hash":"600911285b32fd739ab2291d3218cb6dbd45e6821cb13154ae45181e249433ed","question":"[header] How to buy a belt [title] Find several pairs of pants that you are likely to wear with the belt. [step] Look at the inside tag to determine what the waist size is. For example, it may list 30 x 32 inches, in which case 30 inches is your waist size.\n\nA. Look inside the belt compartment to determine what length you will need to purchase. [title] Pick out a length that will cover your hip.\nB. Decide what style you would like and wear it with sturdy t-shirts, short shorts, and cute boots. You may also want to show a prior article of clothing for inspiration or to show off your newly-laid back.\nC. Find pairs of pants that are sizing up to 70 inches, and find which ones you like. [substeps] What makes pants so cool is not the functionality of the pants, but because they are worn.\nD. [title] Measure your waist with a fabric measuring tape if your tag is missing or doesn't list the waist size in inches. [step] Wrap the tape around your mid-section, right where your belly button is.","prompt":"[header] How to buy a belt [title] Find several pairs of pants that you are likely to wear with the belt. [step] Look at the inside tag to determine what the waist size is. For example, it may list 30 x 32 inches, in which case 30 inches is your waist size.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6356,"error":null,"output":"D"}
{"question_id":"hellaswag:00008775","item_index":96,"attempt":0,"prompt_hash":"38f189076f214c345b0fc88ce2dc41d1cb9757b35c830f8958a2ce4a53a4f7f7","question":"[header] How to wish someone a happy retirement [title] Choose a card that is sentimental to emphasize that you enjoyed working with the person. [step] One approach to selecting a card is to find a message that reflects how you felt about working with the person. If they had an impact on you, you can let them know that they made your time at work better and that you will miss them.\n\nA. [substeps] You can thank the person for his or her work with the card by saying that you're so pleased you can't wait to talk again. You can also try a card that is optimistic, inspiring, or sentimental.\nB. [title] Purchase a comedic card if the person likes to joke around. [step] Workplace banter may be something you will remember them for, in which case a funny card makes an appropriate choice.\nC. A message can be easy to catch when your email is filled with snapchats or silly jokes regarding the person's accomplishments. [title] Brighten the recipient's day.\nD. [substeps] Perhaps the person left your office on a bad day and there was a souvenir on their desk. Or maybe you laid behind the desk feeling deeply over that person's work on the good days.","prompt":"[header] How to wish someone a happy retirement [title] Choose a card that is sentimental to emphasize that you enjoyed working with the person. [step] One approach to selecting a card is to find a message that reflects how you felt about working with the person. If they had an impact on you, you can let them know that they made your time at work better and that you will miss them.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6403,"error":null,"output":"B"}
{"question_id":"hellaswag:00007716","item_index":97,"attempt":0,"prompt_hash":"7c58e674aa588e857a5084dc43042c26a236d19012c84b62a243f91f224332f6","question":"[header] How to reduce toddler food throwing messes [title] Put a bib on your child. [step] A bib can prevent you from having to re-clean clothes because of messes created when eating. Make it a habit of putting a bib on your toddler before every meal.\n\nA. There are a variety of different bib types that are available at department stores or online. Feeding bibs are larger and are specifically made to keep messy toddlers clean.\nB. [substeps] Make sure the bib is tightly fitted on your child's face and neck. Only put on bib periodically to prevent spills.\nC. [substeps] Bib tops look great when you walk your child home from school and give your toddler a bib. Bib tops help them to maintain their weight-lifting skills and combat vomit and messes.\nD. Most restaurants offer bib holders that allow you to feed your child by the bib. These areas can be easily cleaned with a vacuum.","prompt":"[header] How to reduce toddler food throwing messes [title] Put a bib on your child. [step] A bib can prevent you from having to re-clean clothes because of messes created when eating. Make it a habit of putting a bib on your toddler before every meal.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6402,"error":null,"output":"A"}
{"question_id":"hellaswag:00005145","item_index":98,"attempt":0,"prompt_hash":"5a691469b3ea634590e265fcaf4f911467fe5d1333b59f02093af2707364fd37","question":"[header] How to make lavender breath freshener [title] Choose the best dry culinary lavender. [step] Many grocery stores sell lavender stems either in the spices or vegetable aisle. Farmer's markets are another great source.\n\nA. You will need an estimated 1 tablespoon (14. 8 ml) of dry lavender for this recipe.\nB. Call a local yard sale if your mom packed your garden with lemon/lime or other organic herbs and seeds. [title] Scoop out a cup of air freshener in a jar.\nC. These produce more fragrant produce, rather than a more pungent source of odors. The moisture of the berries and herbs causes their leaves to produce more harsh, bitter scents.\nD. But if your home is very dry or stuffy, you should hire herbs that are hardy. [substeps] If this is a decision for you, go for the organic organic herbs, as they'll be almost entirely natural and can become the freshest scent.","prompt":"[header] How to make lavender breath freshener [title] Choose the best dry culinary lavender. [step] Many grocery stores sell lavender stems either in the spices or vegetable aisle. Farmer's markets are another great source.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6402,"error":null,"output":"A"}
{"question_id":"hellaswag:00008409","item_index":99,"attempt":0,"prompt_hash":"90790b592f183f5491d992ee75b44e1e6844aa26b00df69b57684d3237d1b01d","question":"[header] How to build your engagement ring online [title] Check the latest trends of engagement rings. [step] It's not hard to find what is latest. You can either check engagement rings images or visit the wedding blogs.\n\nA. If you are using the internet to help you find wedding rings, be sure to check out what is currently popular. You can also browse wedding blogs just to help you get ideas, both to lower your cost and to get the best results.\nB. Find out what's new in the market, what's getting hurt, and where. You can also use a search engine to seek anything related to engagement rings or wedding bands.\nC. The deal site ebay (charactergroup) hosts sites like this, which usually provide updated information about the latest advances in engagement ring design and engagement rings in your country. [substeps] You can schedule an appointment to inspect engagement rings in person and obtain an estimate as early as possible.\nD. In addition, you can browse some fashion blogs that allow you to get an idea about what is hot nowadays. If you are following celebrities in your routine life fashion, you should take a look on the engagement rings of celebrities, either.","prompt":"[header] How to build your engagement ring online [title] Check the latest trends of engagement rings. [step] It's not hard to find what is latest. You can either check engagement rings images or visit the wedding blogs.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6402,"error":null,"output":"A"}
{"question_id":"hellaswag:00001968","item_index":100,"attempt":0,"prompt_hash":"986817862d2656c8455fd16eafb0e53187d05d01a5455cb465d900ce4d608197","question":"A man bends at the knees in front of a barbell. he\n\nA. lifts it slowly, bringing it to his chest.\nB. is using a blue tool a small.\nC. stands up to place his hands on the barbell.\nD. does bends and stuff on the ground.","prompt":"A man bends at the knees in front of a barbell. he","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6403,"error":null,"output":"C"}
{"question_id":"hellaswag:00002417","item_index":101,"attempt":0,"prompt_hash":"ddcf7289ad45b12d3279739352a0ae7fcbc2228df6e83f53d8c942239eed9194","question":"A man sitting outside a house plays a song while playing the guitar and harmonica at the same time. he\n\nA. thumps his hand on the guitar to make a beat.\nB. carries the instrument putting it down very upset.\nC. performs so fast the song starts to blur the camera.\nD. sits for a bit to continue to play the piano in a strange seated position while other individuals stand outside to listen.","prompt":"A man sitting outside a house plays a song while playing the guitar and harmonica at the same time. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6402,"error":null,"output":"A"}
{"question_id":"hellaswag:00008836","item_index":102,"attempt":0,"prompt_hash":"a06d2fe5d3b1305a8b109d553e372020da925ec56d93ecad1ad92854332bdfb1","question":"[header] How to use sambal oelek [title] Use it wherever you'd use hot sauce. [step] Sambal oelek goes wherever hot sauce goes. Try using it to dip fries, as a topping for cooked meat, or to spice up your tomato sauce.\n\nA. If you like to use hot sauce when you, try whisking in some sambal oelek instead. [substeps] A little goes a long way, so even a half teaspoon or so will add lots of heat to your meal.\nB. [title] Use sambal oelek to stir fry vegetables. [step] Mix up foods like cauliflower or cranberry with sambal oelek to make a simple tomato sauce to use on veggies.\nC. You'll typically apply it to elbow-deep dip and serve it with other vegetables and meats. [substeps] Alternatively, you can use it as a base base to make a pre-mixed sauce.\nD. It also makes meat delicious! [title] Make your own sambal oelek. [step] This is a delicious sauce! Here are some ideas for making it : [substeps] Grate your tomatoes into a cheese bowl and combine equal parts vinegar and water.","prompt":"[header] How to use sambal oelek [title] Use it wherever you'd use hot sauce. [step] Sambal oelek goes wherever hot sauce goes. Try using it to dip fries, as a topping for cooked meat, or to spice up your tomato sauce.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6400,"error":null,"output":"B"}
{"question_id":"hellaswag:00000199","item_index":103,"attempt":0,"prompt_hash":"3fec612847e8dac336543fe229d95fa476f5afa08d7e080ff82c3cec8fcfe0f0","question":"A first person view is seen of a man riding a riding lawn mower. He takes turns quickly, mowing the lawn. he\n\nA. continues until the entire yard is mown.\nB. never stops, trampling the grass and clipping it back.\nC. moves back and fourth doing several other tasks while looking to the camera.\nD. backs up slowly to cut the lawn.","prompt":"A first person view is seen of a man riding a riding lawn mower. He takes turns quickly, mowing the lawn. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6401,"error":null,"output":"A"}
{"question_id":"hellaswag:00006079","item_index":104,"attempt":0,"prompt_hash":"eaa654caf33d11da230dd2a610f746282901a679734a51179b4029c74a64d96f","question":"[header] How to store watermelon [title] Store whole watermelons at room temperature. [step] Putting a whole watermelon in your fridge can actually reduce its nutritional value. If you're not planning on cutting up your melon right away, it's best to store the whole melon at a room temperature, out of direct sunlight.\n\nA. Floating in your refrigerator may create more ice cubes than it needs to. [title] Keep whole squash in a freezer-safe container with ice rinds.\nB. [title] Assemble the watermelon halves and core. [step] Peel the flesh from the watermelon using a knife and discard the pulp.\nC. [substeps] You should still cut up your watermelon, or freeze it, within two weeks of buying the whole fruit. [title] Wrap the cut end of the watermelon with plastic.\nD. This will save the appearance of the fruit pulp on your frozen watermelon. [title] Wash and dry your watermelon.","prompt":"[header] How to store watermelon [title] Store whole watermelons at room temperature. [step] Putting a whole watermelon in your fridge can actually reduce its nutritional value. If you're not planning on cutting up your melon right away, it's best to store the whole melon at a room temperature, out of direct sunlight.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6400,"error":null,"output":"C"}
{"question_id":"hellaswag:00007565","item_index":105,"attempt":0,"prompt_hash":"374d814b53672ae97b3ee5cb4d442324999fdd4ec2749fbadc807522d53bc49c","question":"[header] How to take care of naughty kids [title] Draft a set of rules. [step] This set of rules should be written with consideration of your child's age. Younger children will need simple, straightforward rules, while older children can begin to understand more complex rules, that may be flexible depending on the situation.\n\nA. [title] Create a habit of doing everything the child wants to do every day for several weeks. [step] Show your children what their responsibilities are and how to find meaning by breaking these goals.\nB. [substeps] For example, if your child is very young, rules for the following are up to age 7 or below: no yelling, no fighting, no rough play, and no negative discipline. If you're going to have trouble deciding when to discipline a little child, don't worry about being specific.\nC. [substeps] Consider how the rules are designed to determine how much overspray is required. Infants and toddlers should know how to clean out their pockets and bottles, while toddlers should understand how to properly sanitize their skin.\nD. The list should prioritize those rules that are based on the naughty behaviors your children is demonstrating. [substeps] For example, if your child gets aggressive when they don't get what they want, then you should draft a rule that makes this behavior strictly forbidden.","prompt":"[header] How to take care of naughty kids [title] Draft a set of rules. [step] This set of rules should be written with consideration of your child's age. Younger children will need simple, straightforward rules, while older children can begin to understand more complex rules, that may be flexible depending on the situation.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6399,"error":null,"output":"D"}
{"question_id":"hellaswag:00001766","item_index":106,"attempt":0,"prompt_hash":"a2299f889598c10e8bea87dab81ad4c5574ee6af2f7eeee7607a25e6bcf58a08","question":"A group of people are seen wandering around as well as a man speaking to the camera. a close up of a person running\n\nA. is seen as well as people playing a game up on the horse's back.\nB. down a track is shown as well as the man adjusting clothing and other outdoors tasks.\nC. is shown in a public place.\nD. bungee jumps into the pool.","prompt":"A group of people are seen wandering around as well as a man speaking to the camera. a close up of a person running","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6401,"error":null,"output":"B"}
{"question_id":"hellaswag:00001438","item_index":107,"attempt":0,"prompt_hash":"9dbc190ee4cac9a1bb82a0ff17d258aa68122509115830c9a8526b673bda9eb7","question":"The woman begins to emulate the man's actions, moving her arms and dipping her head under the water. the man\n\nA. then appears out of nowhere and begins to hold the woman.\nB. shows her how to swim outward by doing so himself.\nC. reaches out and she jumps in, submerged at the waist.\nD. licks his lips as the woman moves under the water in the same manner.","prompt":"The woman begins to emulate the man's actions, moving her arms and dipping her head under the water. the man","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6400,"error":null,"output":"A"}
{"question_id":"hellaswag:00004887","item_index":108,"attempt":0,"prompt_hash":"b016f752872a9f29a5ebe6f0b477d40c32ff49b213ba20f488a0e26515e07ee9","question":"[header] How to highlight hair [title] Choose the right color. [step] For highlights, you'll either need to use bleach or a dye that's lighter than your hair color, depending on your hair's color and condition. It's best to aim for a color that is one to two shades lighter than your base color.\n\nA. Essentially, you want your hair to be slightly lighter than your base color. [substeps] It will take more than a few dye sessions to achieve your desired color.\nB. [substeps] The amount of highlighting you do depends on your hair type. 1.5 to 0.5 shades lighter than your base color usually goes well.\nC. Going too light can create an unnatural, stripy effect. If you have the option, go for a box that comes with a toner.\nD. For highlights, opt for a cool blond or caramel tint. [substeps] In addition to highlighting hair, opt for an orange or teal color, if you want as your highlights color.","prompt":"[header] How to highlight hair [title] Choose the right color. [step] For highlights, you'll either need to use bleach or a dye that's lighter than your hair color, depending on your hair's color and condition. It's best to aim for a color that is one to two shades lighter than your base color.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6399,"error":null,"output":"A"}
{"question_id":"hellaswag:00002052","item_index":109,"attempt":0,"prompt_hash":"c967d69f24efa19e6ad6c42193e3beb01d9fc31ae226733d2b7f777c466e68ed","question":"Various shots of a city are shown followed by a girl playing violin and many other watching. another girl\n\nA. is seen speaking to the camera and then moving another to another side.\nB. steps in and joins her by singing and several others join in to sing as well.\nC. continues playing with her violin while looking around and pausing to dip it into a bucket.\nD. plays instruments with the camera continuously showing them while others walk around and do the same.","prompt":"Various shots of a city are shown followed by a girl playing violin and many other watching. another girl","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6399,"error":null,"output":"B"}
{"question_id":"hellaswag:00009845","item_index":110,"attempt":0,"prompt_hash":"e1bebbe8635b9ad249c46404e6fa8f97167dbe71b1e13d03c3ec26a885e98e9c","question":"[header] How to become cpr certified [title] Identify reasons to get certified. [step] There are many reasons to get certified in cpr. Some of the most important ones include : [substeps] You can save lives-it teaches you to be aware of your surroundings and recognize signs of cardiac arrest and other debilitating situations.\n\nA. You can defend yourself if someone is choking on you or you feel the need to terminate the cpr training unit. [title] Publish your requirements for cpr training.\nB. You are better equipped to assist others in need. Studies have shown that cpr certified individuals are more likely to lend a hand in time of need.\nC. People in cpr would benefit from volunteering and on medical training programs. Though there are all sorts of certifications awarded to recognized medical students, there are only a few ways that you can get a certification.\nD. You can help cover up fires-make medical emergency medical diagnoses happen quickly and effectively. You can encourage people to stop smoking and to use breathing purifiers such as artonite and prozac that protect people at all times.","prompt":"[header] How to become cpr certified [title] Identify reasons to get certified. [step] There are many reasons to get certified in cpr. Some of the most important ones include : [substeps] You can save lives-it teaches you to be aware of your surroundings and recognize signs of cardiac arrest and other debilitating situations.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6399,"error":null,"output":"B"}
{"question_id":"hellaswag:00008900","item_index":111,"attempt":0,"prompt_hash":"c2b167583cb70413e6374485e802843651773d85a7e9af6416598b21dc7ae11b","question":"[header] How to fit in [title] Identify the group that you want to fit in with. [step] Most likely this will be the popular \" in \"-crowd, but think of specific words that describe the group. This will help you better visualize and organize the things you will need to do to fit in.\n\nA. Here are some common ones : [substeps] Dress up. Most social events will require you to dress up, so be sure to dress in layers.\nB. For instance, if it is \" in \", \" fi \", or \" i'm in \", then consider moving on to the more formal ones. [title] Choose a group who fits in more with your current group.\nC. [title] Consider the people that you'll attend. [step] This can include your friends, parents, new students or people that you might not have considered.\nD. [substeps] Use this template: the popular kids are _ _ _ _ _ _ _ _ _ _. They are best described as _ _ _ _ _ _ _ _ _ _.","prompt":"[header] How to fit in [title] Identify the group that you want to fit in with. [step] Most likely this will be the popular \" in \"-crowd, but think of specific words that describe the group. This will help you better visualize and organize the things you will need to do to fit in.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6399,"error":null,"output":"C"}
{"question_id":"hellaswag:00001933","item_index":112,"attempt":0,"prompt_hash":"4caa94ee4d81cd0ed215238fb1703ff6021e9dc6fd15121f129fc93f06cadc16","question":"We see a rowing machine in a exercise room. a lady\n\nA. picks up a machine and lifts it to her shoulder, then drops it.\nB. starts putting her feet in the turning machine.\nC. sits on the seat and pulls the cord of the rowing machine.\nD. pushes a machine.","prompt":"We see a rowing machine in a exercise room. a lady","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6570,"error":null,"output":"C"}
{"question_id":"hellaswag:00001445","item_index":113,"attempt":0,"prompt_hash":"72a05a97ed51b1ec3f1d5f2a7563fd36068c4fd60217b475ab2ebb18201e0f8a","question":"They are swishing it in their mouths and almost all of them have their hands over their mouths. one of them\n\nA. is screaming at the camera.\nB. spits it out in the sink and the other three are trying to carry on.\nC. has a giant alligator in his hands and he starts pulling on it so that it doesn't bite him as he moves back and forth with the same claw nails.\nD. catches one of the girls in his mouth by her face, she hits him and he flashes a big smile at her while she mystery friends are sitting around looking very intrigued.","prompt":"They are swishing it in their mouths and almost all of them have their hands over their mouths. one of them","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6569,"error":null,"output":"A"}
{"question_id":"hellaswag:00005766","item_index":114,"attempt":0,"prompt_hash":"ad134b2a954eb450fbfbdee2b649b0c9835531c47a36a9b0acd8ab554f1fd136","question":"[header] How to continue breastfeeding after returning to work [title] Prepare yourself. [step] Before returning to work, you need to plan ahead and prepare early. This means building up a stockpile of expressed breast milk (ebm) and establishing the infrastructure for successfully expressing breast milk at work.\n\nA. [substeps] Stockpiling seems to solidify the effective synthesis of a healthy, consistent breastfeeding cycle. Having a fresh supply of ebm, however, can make it difficult to maintain a healthy and productive relationship with your baby.\nB. [substeps] Determine your daughter's ideal age. If your daughter is just six months old, set aside a small amount of time to begin breastfeeding.\nC. If you invest the time and effort to get this done first, it will be less challenging once you are back at work. Build up a stockpile of ebm while still on maternity leave.\nD. Write everything you want to know down on a separate sheet of paper and save it for later or replace it with a handwritten note. [substeps] If possible, plan to take a day off work.","prompt":"[header] How to continue breastfeeding after returning to work [title] Prepare yourself. [step] Before returning to work, you need to plan ahead and prepare early. This means building up a stockpile of expressed breast milk (ebm) and establishing the infrastructure for successfully expressing breast milk at work.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6568,"error":null,"output":"C"}
{"question_id":"hellaswag:00005929","item_index":115,"attempt":0,"prompt_hash":"278bcfec56a1393329935f959bcbcf995eb06b4d9ea061cfe56a27eefe18f131","question":"[header] How to safely use weight loss shakes [title] Find out if weight-loss shakes are safe for you. [step] Research any health issues you suffer from or medications you're taking. Consider your age, height, and weight, as well.\n\nA. The ingredients and nutritional content of many or most weight-loss shakes might make them unsuitable for you. For example : [substeps] Weight-loss shakes might react with diabetes medications, causing your blood sugar to dip dangerously low.\nB. Some products are safe on you and some don't. If you see that for you, shop around.\nC. Consider how long the shakes last for and how many people you will use them with and if there are any health issues that are preventing you from trying to lose weight. [substeps] If you're new to using weight-loss shakes, don't worry, they'll usually work.\nD. May be overwhelmed by too many meals and changes over time. If you feel that you don't mind feeling overwhelmed, you may want to talk to your doctor about a weight-loss shake.","prompt":"[header] How to safely use weight loss shakes [title] Find out if weight-loss shakes are safe for you. [step] Research any health issues you suffer from or medications you're taking. Consider your age, height, and weight, as well.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6567,"error":null,"output":"A"}
{"question_id":"hellaswag:00009204","item_index":116,"attempt":0,"prompt_hash":"915e1e10921c0d5bc8e2e2496cfbd667c903a595a014b7bb3a7a31ed39d1b1cd","question":"[header] How to use a foot scraper [title] Select a foot scraper. [step] There are many types of foot scrapers available that you may want to check out. While the most common is made with a plastic or wood handle and features a double-sided scraper, they also come in clay, glass, metal or electric forms.\n\nA. [substeps] Small hand scrapers are designed to work with very small areas, such as the pads or the toes. Others are made of plastic and can be mounted to your toe like a pair of scissors or a saw.\nB. [substeps] Clay foot scrapers are sometimes referred to as \" floor scrapers. \" these feet prevent the bottom layer of dirt from coming into contact with the surface of the top layer of dirt.\nC. Decide which type will make it easiest for you to pamper those beautiful feet. [substeps] Most foot scrapers have one side that is rougher than the other to help remove corns or thicker calluses.\nD. Sliding the foot scraper along the top of your shoe to get a feel for how it works will save you time and sweat in the long run, but may not be as efficient. Be sure to refer to manufacturers' photos to get a good idea of the ideal scraper.","prompt":"[header] How to use a foot scraper [title] Select a foot scraper. [step] There are many types of foot scrapers available that you may want to check out. While the most common is made with a plastic or wood handle and features a double-sided scraper, they also come in clay, glass, metal or electric forms.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6568,"error":null,"output":"C"}
{"question_id":"hellaswag:00003711","item_index":117,"attempt":0,"prompt_hash":"0b45de29189aa4430d7f7dc0a3a5b9659948d6e79d6ff366c46a774ed0304f81","question":"[header] How to get a behind the ear tattoo [title] Locate different tattoo artists. [step] The most important step in getting your tattoo is finding the right artist. Doing your homework can lead you to an artist whom you trust as well as offers the aesthetic you want for your tattoo.\n\nA. [substeps] Ask a friend to help, or ask your family, to help you find artists who specialize in your artwork. Ask your doctor or therapist if they offer free or reduced-cost tattoos.\nB. [substeps] Ask a family member or close friend for recommendations. You can always just ask someone at a tattoo parlor or online (or through a trusted friend! ) for recommendations.\nC. It will help you become legally licensed as an artist and carry around the tattoo certificate in the event your application is denied. [substeps] Make a list of artists who have experience working with you on tattoo applications.\nD. [substeps] Ask family, friends, and acquaintances for recommendations of tattoo artists in your local area or beyond. Check tattoo magazines and social media accounts of different tattoo artists to see examples of their work.","prompt":"[header] How to get a behind the ear tattoo [title] Locate different tattoo artists. [step] The most important step in getting your tattoo is finding the right artist. Doing your homework can lead you to an artist whom you trust as well as offers the aesthetic you want for your tattoo.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6567,"error":null,"output":"D"}
{"question_id":"hellaswag:00008305","item_index":118,"attempt":0,"prompt_hash":"23d2bbfc8af36127873da6925645679f9e1842caebe0ed1e9899975a83360f39","question":"[header] How to make a chocolate bubble bath [title] Buy unscented bubble bath. [step] You will need to buy bubble bath to create the bubbly, foamy part of the bath. The chocolate will provide the scent and color of the bath, so do not buy a scented bubble bath.\n\nA. Consider buying a bubble bath that is scented with essential oils, natural ingredients, or herbs and that are unscented. [substeps] Read the products you buy and make sure to read the ingredients on the soap bottle.\nB. Instead, look for a bubble bath with as few ingredients as possible. You can find bubble bath at many supermarkets and bath and body stores.\nC. [substeps] You can also try mixing the bubble bath with a few drops of essential oils to create a unique scent. [title] Fill a large glass or plastic bowl with bubble bath-sized scoops.\nD. Buy one in a gallon jug, as opposed to a standard glass bottle. [substeps] Avoid using bubble bath products, chocolates, or other exotic (carbonated) products.","prompt":"[header] How to make a chocolate bubble bath [title] Buy unscented bubble bath. [step] You will need to buy bubble bath to create the bubbly, foamy part of the bath. The chocolate will provide the scent and color of the bath, so do not buy a scented bubble bath.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6569,"error":null,"output":"B"}
{"question_id":"hellaswag:00005609","item_index":119,"attempt":0,"prompt_hash":"0f2cc2b4e926b6193b44874539161cd866a34d0e0237803f5a56747446a93857","question":"[header] How to protect your pearls [title] Use a soft bag/pouch to store them. [step] Okay, so pearls can get scratched really easily. So, to store them use a soft bag or small but soft pouch.\n\nA. [title] Take them out of the box immediately. [step] If they are broken or tear, take them out so they won't get in your way.\nB. These can be stones and shells, plant bark and various other objects. In the end it will be the right size and shape to keep your pearls in.\nC. It is up to you how you use each bag/pouch. You want something that is comfortable and automatic to drop them into, so choose a small bag/pouch that has a low bottom so it can easily get back in after each use.\nD. You can also use a clean soft cloth. What we are basically doing here is avoiding the pearls to get scratched.","prompt":"[header] How to protect your pearls [title] Use a soft bag/pouch to store them. [step] Okay, so pearls can get scratched really easily. So, to store them use a soft bag or small but soft pouch.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6569,"error":null,"output":"D"}
{"question_id":"hellaswag:00000899","item_index":120,"attempt":0,"prompt_hash":"40767186215477b90526630313a9138ea2d7a61c063952fa1c3f88da28b1917a","question":"A white bmw 3 series is in a garage being cleaned by two men wearing safety jackets. once the front\n\nA. of the car is cleaned the men put the back of the car in the back of the van could be seen in the garage.\nB. is cleaned, the men move towards the back spraying the vehicle and come back to spray the rim and go back to the trunk to get all of the spots.\nC. cover is lifted, the rear plate of the car is covered in white mud and a red bucket is shining away.\nD. tires are taken off, the tires get stuck on parts of the car and starts to move.","prompt":"A white bmw 3 series is in a garage being cleaned by two men wearing safety jackets. once the front","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6568,"error":null,"output":"A"}
{"question_id":"hellaswag:00002631","item_index":121,"attempt":0,"prompt_hash":"71b36c56f72b9a526f19146fdd1cd683e4d3a41f510664655ff7546ed4ab3587","question":"A weight lifter is bend to raise a weight sitting on the floor wile a woman watch him. then\n\nA. , the man lifting weight jumps in air while the woman watch him, then he drops the weight on the floor.\nB. he lifts the weight till a third man prepares for weightlifting.\nC. , the man raises the weight to rest on the shoulders, and then above the head.\nD. , the man lift the weight, and then the woman talks.","prompt":"A weight lifter is bend to raise a weight sitting on the floor wile a woman watch him. then","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6568,"error":null,"output":"C"}
{"question_id":"hellaswag:00000278","item_index":122,"attempt":0,"prompt_hash":"4c2d1157590b3a8c14813ab68539e3648a3fd40441da9f561829d982f5f84005","question":"The man jetskis while holding the camera at arm's length. The man is shown waterskiing while pulled by the jetski. the man\n\nA. falls and is briefly submerged.\nB. briefly jettisoned himself trying to follow the jetski.\nC. mows the lawn being pulled by the jetski.\nD. jetskis back and forth while swimming in the water.","prompt":"The man jetskis while holding the camera at arm's length. The man is shown waterskiing while pulled by the jetski. the man","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6567,"error":null,"output":"D"}
{"question_id":"hellaswag:00006108","item_index":123,"attempt":0,"prompt_hash":"69cfa4bb97a12cc0e4e9b8ffa45b6a83a638c85d3a170a9a72e5d053326b700c","question":"[header] How to wear lace up flats [title] Wear a pointed-toe style. [step] Pointed-toe flats are a classic look and pointed-toe lace ups are by far the trendiest design. The sharp toe looks sleek and polished, and these versatile shoes can be easily dressed up or down.\n\nA. Think about renting or making your own shoes, so that you're unique on your and your chosen design. [substeps] Plenty of black, flimsy lace looks great on both heels and flats, and it's also not as noticeable on lots of people as black or sturdy lace-ups.\nB. Many people find them to be a fashionable alternative to heels. Because of their current popularity, you'll find an extremely wide variety to choose from, as well as a range of price points.\nC. They can also be very versatile, creating a fashionable look for a shopper. Accordingly, get fitted for a cute, button-down top with neutral vertical stripes and a slim belt.\nD. [title] Pick peep toes with closed eyeliners and closed toe. [step] This is especially important for somewhere in the middle of the outfit, like a drag show, show or high school dance.","prompt":"[header] How to wear lace up flats [title] Wear a pointed-toe style. [step] Pointed-toe flats are a classic look and pointed-toe lace ups are by far the trendiest design. The sharp toe looks sleek and polished, and these versatile shoes can be easily dressed up or down.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6584,"error":null,"output":"B"}
{"question_id":"hellaswag:00008323","item_index":124,"attempt":0,"prompt_hash":"c592e0a9e7e77c2b9df30d7810bfcab01b27055630419321d88f23de3d069ff1","question":"[header] How to buy shoes [title] Buy shoes from a brand known for quality and comfort. [step] Many well-established footwear brands are known for their reliable, functional footwear. Buy a pair of shoes from one of these brands to make sure that you are getting a high quality product.\n\nA. [substeps] Find the quality label with high quality sizes on the back of the shoes. Confirm that the manufacturer does not use excessive amounts of lotions or lotions in the shoe, as these are designed to discourage blisters and new boots from forming.\nB. [title] Avoid pairs that are flat or flimsy. [step] Flat shoes will be easier to measure if you are buying on a flat surface or a grassy surface that isn't tall enough to stand upright.\nC. [substeps] Buy shoes online to avoid having to buy a pair of shoes that is too high up on your feet. [title] Ask your medical or psychological care caregiver for advice.\nD. Most of these companies have built their reputations on years of good customer service as well, so these shoes are likely to be covered by a generous warranty. [substeps] Find quality footwear brands by reading online reviews.","prompt":"[header] How to buy shoes [title] Buy shoes from a brand known for quality and comfort. [step] Many well-established footwear brands are known for their reliable, functional footwear. Buy a pair of shoes from one of these brands to make sure that you are getting a high quality product.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6584,"error":null,"output":"D"}
{"question_id":"hellaswag:00006387","item_index":125,"attempt":0,"prompt_hash":"b3fbb5bd60ad19a4b7fd452c4c8ffa69fb2fe4323f9c874188bbd9b57e899790","question":"[header] How to care for a sick frog with red leg disease [title] Look for signs of lethargy. [step] You may notice your frog acting tired or being inactive if they are suffering from red leg disease. Always monitor your frog's level of activity to learn what is normal for them, allowing you to detect any signs of illness.\n\nA. [substeps] If you notice your frog looking agitated or lethargic, may go up to the frog and prevent it from lying down if you fear illness. Do not attempt to pick up the frog or pet it.\nB. If you think your frog isn't as active as it normally is, you may want to take it to your veterinarian to check for any medical problems. [substeps] Frogs with red leg diseases will move very little throughout the day.\nC. [substeps] If your frog's activity is decreasing or you notice them acting lethargic, it could mean that they have red leg disease. Runny droppings may also be a sign of illness.\nD. If you suspect your frog has red leg disease, look out for any changes in behavior that might indicate the disease. These changes include : [substeps] Loss of appetite diarrhea loss of appetite redness or irritation [title] Examine your frog's eyes for redness.","prompt":"[header] How to care for a sick frog with red leg disease [title] Look for signs of lethargy. [step] You may notice your frog acting tired or being inactive if they are suffering from red leg disease. Always monitor your frog's level of activity to learn what is normal for them, allowing you to detect any signs of illness.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6585,"error":null,"output":"B"}
{"question_id":"hellaswag:00000793","item_index":126,"attempt":0,"prompt_hash":"f9ade0e1086555b118a4bb2f6af8fc4c80c600c54775261c998d127c320ebf92","question":"A man is shown stretching before engaging in a tennis match in front of a crowd. He then speaks to the crowd amongst flashing cameras through a microphone. the man\n\nA. then resumes speaking to the crowd and pointing off into the distance.\nB. returns to talking to the crowd.\nC. finishes his match and runs a towel over his face.\nD. is occasionally shown playing tennis, then goes back to speaking to the group.","prompt":"A man is shown stretching before engaging in a tennis match in front of a crowd. He then speaks to the crowd amongst flashing cameras through a microphone. the man","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6584,"error":null,"output":"A"}
{"question_id":"hellaswag:00007346","item_index":127,"attempt":0,"prompt_hash":"678fa87413635600fbe8d0b003a2d2e04398bae1eff97b7a6ca4ae92e49e15ef","question":"[header] How to remove whitesmoke [title] Click on \" start, \" then click on \" control panel. [substeps] If using windows 8, position your mouse at the right edge of the screen, click on \" search, \" then search for and select \" control panel. \" [title] Click on \" uninstall a program \" or \" add and remove programs.\n\nA. [step] Check the box beneath \" ad helps remove whitesmoke entirely, \" then click on \" uninstall that program. \" if the \" add and remove programs \" box pops up, select \" not all programs.\nB. [step] The program will remove whitesmoke's seal from windows 8. [title] Follow the in-panel instructions for the program to uninstall.\nC. [step] \" all of the programs currently installed in windows 7 will be checked off. [substeps] You can also delete programs that you want to remove from the control panel.\nD. [title] Search for \" whitesmoke toolbar, \" and select the option to remove the program from your computer. [substeps] In some cases, whitesmoke may only have affected your internet browser settings.","prompt":"[header] How to remove whitesmoke [title] Click on \" start, \" then click on \" control panel. [substeps] If using windows 8, position your mouse at the right edge of the screen, click on \" search, \" then search for and select \" control panel. \" [title] Click on \" uninstall a program \" or \" add and remove programs.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6585,"error":null,"output":"D"}
{"question_id":"hellaswag:00000084","item_index":128,"attempt":0,"prompt_hash":"1ac09568333b04ecc4b61e67516ae58bd2f0534c4791cec1c52f6d283287c1dc","question":"We see a man in a driveway cleaning snow from his car. the man\n\nA. cleans the snow under his windshield wipers.\nB. wet his car and wipes it down.\nC. moves the snow out of the front bumper of his car and pours gasoline on it.\nD. removes the pieces of the line off his windshield.","prompt":"We see a man in a driveway cleaning snow from his car. the man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6441,"error":null,"output":"A"}
{"question_id":"hellaswag:00007049","item_index":129,"attempt":0,"prompt_hash":"4a25c552a28b663ddb16a4e287c2c2e8bf276ed8c39891ad10db6cec6c6d3732","question":"[header] How to march [title] Stand at attention. [step] There are two possible commands for standing at attention: \" fall in \" is used to either assemble individuals into formation or return the marchers to their original lineup. \" attention \" is a command given during rest position.\n\nA. When someone moves to the side or down the line, the person standing at attention is notified. So if the marchers move back and forth, the onlookers could easily interrupt their direction of travel.\nB. You should respond to both commands by standing at attention. [substeps] Bring your heels together evenly, with the toes slightly pointed outward to form a 45-degree angle between your feet.\nC. \" shoulders up \" is an affirmative and followed by a disrespectful turning of the shoulders or lowering the tailbone and lowering the jaw. \" head up \" is a command given during vigorous activity.\nD. You must stand at attention 24/7 whenever you plan to march using either of these commands. Beyond the formation position, you will take on this command in the order of the drill and the beat of the music.","prompt":"[header] How to march [title] Stand at attention. [step] There are two possible commands for standing at attention: \" fall in \" is used to either assemble individuals into formation or return the marchers to their original lineup. \" attention \" is a command given during rest position.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6437,"error":null,"output":"B"}
{"question_id":"hellaswag:00004344","item_index":130,"attempt":0,"prompt_hash":"a49b59ab8d778413fe9ea5356679fceb068b7c48fb789abd8af15fb1366401b0","question":"[header] How to choose a parenting style [title] Know the 4 styles of parenting. [step] Although there are many ways to be a parent, psychologists often refer to 4 main parenting styles. The styles are authoritarian, permissive, authoritative, and uninvolved.\n\nA. Take some time to learn about each the positive and negative qualities of each style so you can figure out which style will benefit your family the most. It's important to consider your specific child's temperament and tailor your parenting style to suit their needs, incorporating as many authoritative aspects as possible.\nB. [substeps] Authoritative parenting styles tend to focus on being a superior leader and supporting all children. Non-authoritarian parenting styles also require that you always become a one-on-one parent.\nC. The styles focuses on incorporating strong communication, discipline, family beliefs, and rule-breaking into the decision making process. [substeps] Discipline refers to your work ethic, ability to follow through and do what you are told, and qualities you and your children are capable of developing in your own life.\nD. [substeps] Authoritative means to act in the best and most mature ways, while authoritative means to assert authority (i.e., assertiveness when something goes wrong). Authoritative encompasses one's own way of taking and retaining power and the needs of others.","prompt":"[header] How to choose a parenting style [title] Know the 4 styles of parenting. [step] Although there are many ways to be a parent, psychologists often refer to 4 main parenting styles. The styles are authoritarian, permissive, authoritative, and uninvolved.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6438,"error":null,"output":"A"}
{"question_id":"hellaswag:00001171","item_index":131,"attempt":0,"prompt_hash":"879c02fa7b75544e6ca895f452bc51ebba6704917f1db5bda4f116fe27ec6ead","question":"An old black and white clip plays and a tall man starts running towards a pole that is up high, and there are a lot of people in the general area doing their own thing or watching him. when the man\n\nA. reaches the pole he jumps very high, clears the pole, lands on the matting below it and people cheer while he gets up and runs back to the area he started.\nB. reaches his pole it comes down with such a little weight, it doesn't even break his fall.\nC. reaches the pole he looks at it and then continue to run towards it till he gets to shoulders height.\nD. gets to the pole and says something, some of the people start jumping beside him while some of them go to other areas.","prompt":"An old black and white clip plays and a tall man starts running towards a pole that is up high, and there are a lot of people in the general area doing their own thing or watching him. when the man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6440,"error":null,"output":"A"}
{"question_id":"hellaswag:00003441","item_index":132,"attempt":0,"prompt_hash":"3d0ebc78583e698844ab68911ed6e0bc1ebdf6561f1a3a0f17d564fbac29750e","question":"[header] How to ignore people you don't like [title] Stay away from the person. [step] Avoidance is sometimes the easiest way to ignore someone. If someone is getting on your nerves, try to keep your distance as much as you can.\n\nA. [substeps] Walk away if someone enters your social circle. If that person waves at you and then leaves, stay where you are.\nB. Try to stay away as much as possible. When you will be around this person, pretend you are not around them.\nC. [substeps] You can avoid places they're likely to hang out. If an annoying co-worker always eats lunch at noon, try eating lunch outside of the office or taking a later lunch.\nD. If the person is annoying you and you didn't hear anything from them in a minute, don't even like them. [substeps] For example, don't visit a new shop when it's closed.","prompt":"[header] How to ignore people you don't like [title] Stay away from the person. [step] Avoidance is sometimes the easiest way to ignore someone. If someone is getting on your nerves, try to keep your distance as much as you can.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6439,"error":null,"output":"B"}
{"question_id":"hellaswag:00005199","item_index":133,"attempt":0,"prompt_hash":"a6c25a01c5d8d166dcf1dcfdba950d1379ceeddd3b6e472f0c091c67648e9e7d","question":"[header] How to restore thick dried out nail polish [title] Get acetone from a beauty supply shop. [step] Most beauty shops sell acetone. You may also be able to find it in a drug store that sells beauty supplies.\n\nA. [title] Cut the end off of your normal nail with a nail file. [step] Not only do these edges make the polish tacky, but they leave behind a very nasty odor.\nB. Do not dilute it in water, as it is sodium bicarbonate (acetone) which can be harmful if spilled. Make sure you check with your specialist if you are unsure.\nC. If you do not have acetone on hand, you can easily apply acetone to your nails yourself. [title] Hold your hand out to the side.\nD. Once you have your acetone, add a few drops to a container. [substeps] Acetone is acidic, so you will need a container it will not eat through.","prompt":"[header] How to restore thick dried out nail polish [title] Get acetone from a beauty supply shop. [step] Most beauty shops sell acetone. You may also be able to find it in a drug store that sells beauty supplies.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6437,"error":null,"output":"D"}
{"question_id":"hellaswag:00008916","item_index":134,"attempt":0,"prompt_hash":"97da67580d0a041e16e3627e7efab5edab81bab9fa508dafcd866acdeab873c6","question":"[header] How to feel comfortable having small muscles [title] Concentrate on health--not just looks. [step] Big, masculine muscles are considered indicators of strength and health. However, while there is some truth to this, many very strong and healthy individuals do not look like body builders.\n\nA. [substeps] Look at a variety of athletes; the type of muscles develops depends quite a bit on the sport trained for. Weightlifters will look quite different than divers, speed-skaters, or professional dancers.\nB. In contrast, many people look like active runners or cross-legged jumpers. [title] Do not feel sad that you are fat.\nC. This doesn't mean that you should feel bad for taking the steps to look great every day. [substeps] Water can help you to have a toned body.\nD. While these body builders are ready for aggressive physical activity to improve health, they also do not have massive muscles that can do heavy lifting. [substeps] People develop big muscles through many factors, such as, strength training, muscle tone, contact exercises, and cardio.","prompt":"[header] How to feel comfortable having small muscles [title] Concentrate on health--not just looks. [step] Big, masculine muscles are considered indicators of strength and health. However, while there is some truth to this, many very strong and healthy individuals do not look like body builders.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6438,"error":null,"output":"A"}
{"question_id":"hellaswag:00006209","item_index":135,"attempt":0,"prompt_hash":"e9853fd8f89d397aa83bb86a3a535d7e8ca3196e2f6f7058501da9318c167969","question":"[header] How to stop one bridesmaid from stressing out the others [title] Consider the bridesmaids' perspectives. [step] Present the issue to them individually and ask each one how they feel about the situation privately. Each bridesmaid will have a different perspective.\n\nA. [substeps] The theme of the wedding should be about the bride's feelings. Generally, brides do not think about their parents so much, but divorce is also a big deal.\nB. Asking yourself what their perspectives are can lead to hurt feelings. You need to say this aloud and with tears.\nC. The different perspectives will help you see the issue more clearly and may help you decide how to resolve the dilemma. [substeps] For example, say, \" it has been brought to my attention that some bridesmaids are having issues with one of the bridesmaid.\nD. What's with it though? Maybe they've been stressed so often they find their own ways to cope with the stress. What's upsetting them about this experience? How can you help? [substeps] Not only is it okay for some individuals to be under stress, but it's also okay for you too.","prompt":"[header] How to stop one bridesmaid from stressing out the others [title] Consider the bridesmaids' perspectives. [step] Present the issue to them individually and ask each one how they feel about the situation privately. Each bridesmaid will have a different perspective.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6437,"error":null,"output":"C"}
{"question_id":"hellaswag:00004824","item_index":136,"attempt":0,"prompt_hash":"f83b8afa65cada899c8a7101be63110897bb6dcb1b29797695cdc476ff5233dd","question":"[header] How to warp an object in adobe illustrator [title] Open an existing or new document in adobe illustrator. [step] Use the \" place \" option under the file menu to import an object, or create an object using the shape builder tool or the rectangle tool. [title] Make changes to the color, border, size or style using the tools in the tool panel.\n\nA. [step] Click on the object you have created. Ensure you have selected the right layer in the layer panel before you start to warp your object.\nB. [title] When done, click \" file \" and select \" export \" in the dialogue box. [step] In the \" insert new font \" screen, select \" picture set \" from the drop down menu and click \" export \" to make the name of the font that just opened.\nC. [title] Create a new document with formatting techniques you would use when creating an existing one, such as illustrating pictures with tabs. [title] Blend the picture into the background using the combos tool.\nD. [step] In the layer palette, click \" be quick and add \". Choose the size and border bar and then click on the selected location in the palette.","prompt":"[header] How to warp an object in adobe illustrator [title] Open an existing or new document in adobe illustrator. [step] Use the \" place \" option under the file menu to import an object, or create an object using the shape builder tool or the rectangle tool. [title] Make changes to the color, border, size or style using the tools in the tool panel.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6437,"error":null,"output":"A"}
{"question_id":"hellaswag:00001424","item_index":137,"attempt":0,"prompt_hash":"20b60cd625697c75451c6eb4b9aac2757165b54e1aa84d2c1abe221d60af6272","question":"A man in red turban stands in a kitchen. He pours some ingredients into a glass. He explains his methods holding a spoon. he\n\nA. then finishes the drink with a garnish.\nB. holds the spoon in his hands and lifts it up to show the proper position.\nC. pours the ingredients into a bowl at the counter.\nD. adds salt and peppers into the glass as well.","prompt":"A man in red turban stands in a kitchen. He pours some ingredients into a glass. He explains his methods holding a spoon. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6437,"error":null,"output":"A"}
{"question_id":"hellaswag:00007462","item_index":138,"attempt":0,"prompt_hash":"c547fec6cb759c53c8b8e639bc97650e3d5b9fc08541f33a7359eff1af3e5fe9","question":"[header] How to have a balanced life [title] Maintain a schedule. [step] One way to ensure a balanced life is to schedule your time. If you're spending all of your time at work or most of your free time doing one activity, it's hard to maintain balance in all areas of your life.\n\nA. If you only do one activity a day or you have to work every day, having a schedule is hard to maintain. Take steps to keep yourself apprised of your schedule.\nB. What you do in all that time matters just as much as what you do in your daily life. Designate some time for yourself each day, whether you're at work, at home, or while having dinner with friends.\nC. A schedule can help you to balance your time in various areas that are important to you. Maintaining a schedule can help you accomplish goals and create balance in how you spend your time.\nD. Break out your schedule into two or three main categories: \" work \" and \" life \". [substeps] Choose one of the two categories for the day.","prompt":"[header] How to have a balanced life [title] Maintain a schedule. [step] One way to ensure a balanced life is to schedule your time. If you're spending all of your time at work or most of your free time doing one activity, it's hard to maintain balance in all areas of your life.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6439,"error":null,"output":"C"}
{"question_id":"hellaswag:00005894","item_index":139,"attempt":0,"prompt_hash":"ab104b1d5c6a8e522531da96c360fedd1c8b622606b8bd6899e8cb4370932863","question":"[header] How to get a uk visa [title] Check if you need a visa. [step] Even if you are not a british citizen, there are several groups of travelers who will not need a visa in order to enter the uk. Getting a visa can be a long and expensive process, so if you don't need one, it's probably better not to bother.\n\nA. [title] Decide what kind of accommodation you are willing to make in the uk. [step] Visa websites allow you to record many of the items for which you are willing to travel.\nB. These groups are mainly usps, mps, and sas. Some visa options include : [substeps] Https: // consumer.\nC. [substeps] In australia, for example, you will need a passport from a consulate that has approved the application of the visa application. For example, if you need a foreign passport, you will need a passport that is valid for about four years.\nD. [substeps] Certain groups have free passage into the uk. Citizens of other european union nations, non-eu members of the european economic area, switzerland, and members of the overseas countries and territories (oct) do not need a visa.","prompt":"[header] How to get a uk visa [title] Check if you need a visa. [step] Even if you are not a british citizen, there are several groups of travelers who will not need a visa in order to enter the uk. Getting a visa can be a long and expensive process, so if you don't need one, it's probably better not to bother.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6437,"error":null,"output":"D"}
{"question_id":"hellaswag:00003077","item_index":140,"attempt":0,"prompt_hash":"3e80cfee6932f1ce1821e214941008c1a95f9ee6a2bb50645db142bccd5c57aa","question":"A coach explains to a group of teen basketball players in a gym. the teens\n\nA. make numerous blakes into obstacles.\nB. practice, and kick the ball as when it goes down the court.\nC. are aligned in two lines, then they pass the ball each other to threw the ball to the basket.\nD. grab their rods and train in the same formation.","prompt":"A coach explains to a group of teen basketball players in a gym. the teens","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6437,"error":null,"output":"C"}
{"question_id":"hellaswag:00005414","item_index":141,"attempt":0,"prompt_hash":"c82883728eccc0a214c2b01f2a0846a034df81a8271bdb8c3aa7cca8784509c5","question":"[header] How to dress up a graphic tee [title] Wear a statement jacket. [step] Pairing your graphic tee with a jacket that commands attention tones down the informal vibe a graphic tee can give off. For example, try a suit jacket with a micro-pattern, or go even bolder with a long, leopard-print trench coat.\n\nA. [substeps] A loose-fitting, belted top coat can give your graphic tee a pick-me-up. Try pairing a graphic tee featuring a band's name or logo with a nice leather jacket.\nB. [title] Choose a graphic tee that leans towards the torso. [step] Graphic tees with sleeves are for a more casual look, whereas a graphic tee without sleeves is a bit too formal for formal events.\nC. [substeps] Over the shoulder dressier shirts look great with graphic tee tops, but a bold bolder color often draws more attention to the t-shirt as a sharp contrast against the jacket. [title] Select a graphic tee to make it a statement.\nD. A more luxurious look can occasionally be stripped away in the summer months and replaced with something revealing to show off your real' roots'. [substeps] Some of the biggest? Super cute and trendy graphic tees include: ts, turtle-neck t.","prompt":"[header] How to dress up a graphic tee [title] Wear a statement jacket. [step] Pairing your graphic tee with a jacket that commands attention tones down the informal vibe a graphic tee can give off. For example, try a suit jacket with a micro-pattern, or go even bolder with a long, leopard-print trench coat.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6439,"error":null,"output":"B"}
{"question_id":"hellaswag:00005238","item_index":142,"attempt":0,"prompt_hash":"80192209628f25d5e2ca078c24b84849fb556e547bad1b2830ea227e53e78dc9","question":"[header] How to throw a knife without it spinning [title] Take the proper grip. [step] Sticking a knife with no rotation is made possible by modifying the standard throwing grip. Wrap your hand loosely around the handle of the knife.\n\nA. If you have metal or faux stainless steel, be sure that it is well oiled beforehand. Additionally, have an assistant with you to support your hand while you throw the knife.\nB. Pinch the handle between your thumb and the length of your middle finger. Place your index finger flat against the spine of the blade at the knife's center of balance.\nC. The thumb is positioned to the side of the handle. Left-handed throwers should not have put their left hand on the handle.\nD. Wrap your other hand around the handle so that there is a flat surface between you and the knife. [substeps] For example, if you are throwing a frozen banana, the banana will end up eventually turning jelly, so you will need to hold the knife in your non-dominant hand while you throw.","prompt":"[header] How to throw a knife without it spinning [title] Take the proper grip. [step] Sticking a knife with no rotation is made possible by modifying the standard throwing grip. Wrap your hand loosely around the handle of the knife.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6437,"error":null,"output":"B"}
{"question_id":"hellaswag:00001273","item_index":143,"attempt":0,"prompt_hash":"fb1108a3841d789af16a14eabbe8d2c29202792dbd61448a25797a7c37c8557f","question":"A man leans over a pool table, cue in hand. he\n\nA. retrieves the cue, introducing the game.\nB. leans over the scoreboard.\nC. hits the white ball, knocking the other balls toward the pockets.\nD. picks up a ball, and hits it with a cue stick.","prompt":"A man leans over a pool table, cue in hand. he","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6436,"error":null,"output":"C"}
{"question_id":"hellaswag:00002158","item_index":144,"attempt":0,"prompt_hash":"28ae6b3d26cf5a9af392fd2784eb33a38f9f05b06d07c64c7b5f40b21f008867","question":"A woman dressed in a blue flowing dress with a blue bodysuit underneath is standing up, and she begins to do a ballet dance routine in the middle of the large dancing floor while the people in the reception area are watching her every move, taking picture and videoing her. when the woman\n\nA. is done dancing she stands there with her arms crossed on her chest and the crowds applauds, and gives her a standing ovation while she takes a few bows.\nB. is done dancing, she lifts her face mask and starts to laugh off into the distance and when she removes the mask, there's a man behind her who is also laughing off camera in a hotel room, then there's the body suits laughing off camera in a hotel room.\nC. is finished with the dance, a man shows up to help and they all both begin to leave the floor, then there is a pause in the choreography.\nD. is done while all the people are still watching, a newscaster wearing a blue dress comes onto the screen and begins talking and showing a newspaper with a logo.","prompt":"A woman dressed in a blue flowing dress with a blue bodysuit underneath is standing up, and she begins to do a ballet dance routine in the middle of the large dancing floor while the people in the reception area are watching her every move, taking picture and videoing her. when the woman","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6234,"error":null,"output":"A"}
{"question_id":"hellaswag:00007085","item_index":145,"attempt":0,"prompt_hash":"05ecbd59cb86ce13de17a418fce327a9ffbfddfe8f20e34feea1efbff6eb7b1f","question":"[header] How to use the apple magic keyboard [title] Turn on the magic keyboard. [step] The power button is on the back edge of the keyboard on the left side. When you press it, the green light will turn on.\n\nA. When it doesn't turn on, press the home button. [title] Press and hold the letter to power on the keyboard.\nB. [title] Look behind the keyboard, near the battery-powered keypad. [step] You should see a red square with the message \" turn off keyboard.\nC. [substeps] To turn on the magic keyboard, press the power button and hold down the power button for a few seconds. Once the lights on the magic keyboard turn on, grab your magic keyboard and press it with two fingers.\nD. [substeps] The magic keyboard has an internal battery that must be charged. If the keyboard doesn't turn on, use the lightning-to-usb cable to charge it now.","prompt":"[header] How to use the apple magic keyboard [title] Turn on the magic keyboard. [step] The power button is on the back edge of the keyboard on the left side. When you press it, the green light will turn on.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6238,"error":null,"output":"D"}
{"question_id":"hellaswag:00000000","item_index":146,"attempt":0,"prompt_hash":"31c078b36b4caeb93ef4f982434665db015fd6bc9e00d1b3796a7920ddde04e4","question":"A man is sitting on a roof. he\n\nA. is using wrap to wrap a pair of skis.\nB. is ripping level tiles off.\nC. is holding a rubik's cube.\nD. starts pulling up roofing on a roof.","prompt":"A man is sitting on a roof. he","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6235,"error":null,"output":"C"}
{"question_id":"hellaswag:00004330","item_index":147,"attempt":0,"prompt_hash":"a5bb661882de6cf7e9c0e8a394047600fe8b59564bd1103416ae8490bd54dd8e","question":"[header] How to make a homemade (natural) protein hair mask [title] Mix the egg yolk and yogurt. [step] In a small bowl, beat the yolk from 1 egg. Next, add 6 tablespoons of plain yogurt, and use a spoon to mix until the ingredients are fully combined.\n\nA. [title] Add the rest of the ingredients, except for the milk and oil. [step] Once the amount of ingredients have been mixed, mix in approximately 2 egg yolks.\nB. [title] Combine the walnuts and coconut oil. [step] Pour the mashed walnuts and coconut oil into a separate bowl and place it in the electric mixer with the speed dial on medium for about 3 minutes.\nC. Place the mixture into a pan on medium heat and cook, stirring occasionally. [substeps] When the egg yolk mixture comes to a boil, stir in 2 tablespoons of powdered sugar.\nD. [substeps] Egg yolk is rich in proteins and fats, so it helps strengthen and moisturize dry, brittle hair. In addition to protein, yogurt contains lactic acid, which helps remove dirt and product buildup while also moisturizing the hair.","prompt":"[header] How to make a homemade (natural) protein hair mask [title] Mix the egg yolk and yogurt. [step] In a small bowl, beat the yolk from 1 egg. Next, add 6 tablespoons of plain yogurt, and use a spoon to mix until the ingredients are fully combined.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6234,"error":null,"output":"D"}
{"question_id":"hellaswag:00002896","item_index":148,"attempt":0,"prompt_hash":"ddb89fd314cf657d0e7b5d53a1b72956d6d87207dd502cb7b06e52ba70260dff","question":"The guy walks through a door he opened. The guy plays a harmonica with a microphone in one hand. The guy plays the guitar. the black amplifier\n\nA. turns off and the guy then pulls off the light.\nB. switches to a higher octave.\nC. is displayed on a table.\nD. film is shown again by itself.","prompt":"The guy walks through a door he opened. The guy plays a harmonica with a microphone in one hand. The guy plays the guitar. the black amplifier","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6235,"error":null,"output":"C"}
{"question_id":"hellaswag:00001615","item_index":149,"attempt":0,"prompt_hash":"0f0b7b7aced173c1168289f05b551ca8fe4cdb6ee4bd330ec64510aaaefafd79","question":"There is a boy sitting on the bed showing off his sunburn and putting cream on it. A friend is shaking his head because of the other guy's sunburn. the friend\n\nA. holds a little pin he is holding up and pushes it in the sunburn.\nB. puts his shirt on, gets up and walks out of the room again and another boy is showing the same sunburn.\nC. puts a bottle of suntan lotion and rubs it on the legs of the man.\nD. lifts his shirt to show his sunburn as well.","prompt":"There is a boy sitting on the bed showing off his sunburn and putting cream on it. A friend is shaking his head because of the other guy's sunburn. the friend","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6236,"error":null,"output":"D"}
{"question_id":"hellaswag:00007653","item_index":150,"attempt":0,"prompt_hash":"ff2a0b007de9eb0bac0ec428d7ff6e3d69faedf9be47c1da4de88eb2038f1523","question":"[header] How to receive a massage [title] Remove your clothes and wear whatever makes you comfortable. [step] There will be a sheet covering your body. The therapist only pulls away the part of the sheet of the part of the body they are massaging.\n\nA. This includes your back, thighs, belly, neck, and the surrounding areas. To do this, you could lie down with legs wrapped around your body.\nB. However, if you wear a skirt, dress code isn't an issue here. It means the massage oil you're applying should be included in your dress code.\nC. Be suspicious if they want to force you to strip. [title] Before the massage, a hot sauna or shower, if possible is nice.\nD. Before the session starts, remove your clothes (including any undergarments), if you want to remove them. [substeps] You will likely be covered in a plastic wrap which you can push away while the massage begins.","prompt":"[header] How to receive a massage [title] Remove your clothes and wear whatever makes you comfortable. [step] There will be a sheet covering your body. The therapist only pulls away the part of the sheet of the part of the body they are massaging.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6234,"error":null,"output":"D"}
{"question_id":"hellaswag:00000924","item_index":151,"attempt":0,"prompt_hash":"4d9d3979495b24aebd1d6dae0ea1d34c7a9ced362d093eb8a2f18e0eaa44ad74","question":"A woman is seen curling her hair while speaking to the camera and leads into her speaking in front of judges. the judges\n\nA. critique her and she runs away happy and skipping out of the building.\nB. then ddle the woman while she records and showing off her hair style.\nC. then strike off balls as well as speaking to one another and wrapping up the interviews.\nD. speak to her and blow dry her hair while she talks that leads into her flipping the hair and giving a sign on the way.","prompt":"A woman is seen curling her hair while speaking to the camera and leads into her speaking in front of judges. the judges","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6236,"error":null,"output":"A"}
{"question_id":"hellaswag:00003733","item_index":152,"attempt":0,"prompt_hash":"67e08c80dde08bf44204a541b090b2e7f19fbe9e731f1af36e43e2681fb886e2","question":"[header] How to be assertive without being aggressive [title] Take a deep breath and smile. [step] By taking a moment to breathe and smile you not only prevent yourself from acting rashly and aggressively, but you release endorphins in your brain. [substeps] Endorphins are released when you smile because your brain has been trained to recognize these muscle movements as positive.\n\nA. If you feel yourself overconfident, relax and focus on projecting confidence. Relaxing mentally and physically can also help prevent you from becoming angry or frustrated.\nB. Endorphins, along with serotonin keep your mood up and stable. Breathing and smiling also relieves stress because it lowers cortisol levels, which is a stress hormone.\nC. In other words, your stomach should double the number of positive, calming endorphins that are in your body. This should get your mind off the tension of anxiety, which is common in human behavioral patterns.\nD. Red hair is the type that goes dormant at the base of your brain which can lead to a decrease in emotion. Both feelings are very normal, but red hair helps eliminate any negative emotions that may reconstitute later in life.","prompt":"[header] How to be assertive without being aggressive [title] Take a deep breath and smile. [step] By taking a moment to breathe and smile you not only prevent yourself from acting rashly and aggressively, but you release endorphins in your brain. [substeps] Endorphins are released when you smile because your brain has been trained to recognize these muscle movements as positive.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6236,"error":null,"output":"B"}
{"question_id":"hellaswag:00000579","item_index":153,"attempt":0,"prompt_hash":"fa3b95a3ee02b6b178adf67c73b089b34c07fe13353e0ac4c3ecc8637bb2d26e","question":"A man is seen speaking to the camera and leads into him holding up various tools and begins putting them on a bike. the man\n\nA. then ties the strings on the bike while still speaking furiously to the camera and is seen pushing himself along the side.\nB. demonstrates how to properly piece together a bike and peddling the bike to show how it works.\nC. tires the tire and then uses a tool to tighten the hole while continuing to speak to the camera.\nD. is now riding the bike around the area while speaking to the camera and holding up various obstacles on the bike.","prompt":"A man is seen speaking to the camera and leads into him holding up various tools and begins putting them on a bike. the man","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6235,"error":null,"output":"D"}
{"question_id":"hellaswag:00008792","item_index":154,"attempt":0,"prompt_hash":"f066550bc44ff5fc728ce9ae376d9f30cf5f268f997adfb42aef084fc268542e","question":"[header] How to pierce your own tragus [title] Consider seeing a professional. [step] Contrary to popular belief, piercing your own ear is neither easy nor safe. Professional body piercers have the experience, equipment, and environment needed to ensure a quick and clean piercing.\n\nA. [substeps] Poorly performed piercings can lead to infection, bleeding, and nerve damage. You need to understand the risks if you want to continue.\nB. [substeps] Piercing your own ear can be more exciting and ease the stress in getting a piercing, but it's also more risky, as ear piercers are very cautious. [title] Check for anatomy or features of your ear.\nC. [substeps] It is always best to see an experienced piercer or spa teacher before you attempt it. You want to make sure you will get a successful piercing.\nD. Their professional repertoire includes hundreds of different brands and methods. It is your body's preference and you should get along with them.","prompt":"[header] How to pierce your own tragus [title] Consider seeing a professional. [step] Contrary to popular belief, piercing your own ear is neither easy nor safe. Professional body piercers have the experience, equipment, and environment needed to ensure a quick and clean piercing.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6235,"error":null,"output":"A"}
{"question_id":"hellaswag:00004002","item_index":155,"attempt":0,"prompt_hash":"0165b91768950932ec51d0d2c2fe11fd25adff5fb8fbaa2564b67aa45dc6d525","question":"[header] How to stop breastfeeding a toddler [title] Begin by learning what to expect when weaning takes place. [step] You will be better prepared to handle any bumps in the road that may occur if you are not surprised by the events that can potentially accompany the weaning process. [title] Learn what to expect from your body during the weaning process.\n\nA. [step] Natural changes will take place and a good comprehension of those changes will help you to perceive them when they do occur as a normal reaction to cessation of breastfeeding. [title] Understand that emotional shifts are very likely to occur.\nB. [step] It is important to be clear how your body reacts during weaning so that you know what to expect once your child arrives. [substeps] You may need to undergo weaning and growth hormone tests to effectively interact with your baby.\nC. [step] Most weaning procedures are fairly simple and will require little to no effort on your part. Your toddler should become comfortable with losing weight during weaning.\nD. [step] Weaning is often abrupt and unpleasant. While your toddler will initially be able to handle the changes, expect a few demanding bumps along the way.","prompt":"[header] How to stop breastfeeding a toddler [title] Begin by learning what to expect when weaning takes place. [step] You will be better prepared to handle any bumps in the road that may occur if you are not surprised by the events that can potentially accompany the weaning process. [title] Learn what to expect from your body during the weaning process.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6446,"error":null,"output":"A"}
{"question_id":"hellaswag:00000760","item_index":156,"attempt":0,"prompt_hash":"c8f6573fe16cc677ab3e3e48ff60a5fda89674815a5ec76a3bc630d5cb4d8083","question":"A man's face is seen close up speaking and leads into him smoking a cigarette. the man\n\nA. then plays a little piano and sits down to smoke away while continuing to speak and play.\nB. continues to take drags out of the cigarette while blowing smoke off into the distance.\nC. continues smoking with his mouth open and leads into him getting up and speaking to another man as well as smoking cigarettes in a conga line.\nD. walls the room with smoke while moving his hands and speaking.","prompt":"A man's face is seen close up speaking and leads into him smoking a cigarette. the man","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6445,"error":null,"output":"B"}
{"question_id":"hellaswag:00002502","item_index":157,"attempt":0,"prompt_hash":"7d02901e0af900b534fd76cdde8225fe452eabf7e8ef6e1277b69fd047471e90","question":"A female weight lifter bends at the knees. she\n\nA. bends her legs at the knees and extends her arms.\nB. lifts a barbell with her shoulders.\nC. lifts a barbell to her chest.\nD. raises her body over a bar and releases the bar in slow motion.","prompt":"A female weight lifter bends at the knees. she","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6445,"error":null,"output":"C"}
{"question_id":"hellaswag:00000455","item_index":158,"attempt":0,"prompt_hash":"813ffd2abc28fa81108d6d1f374e93983c666670f4f19ff03a967e499f8b0a4b","question":"The men begin putting a boat on the back of a truck. The men tie the boat down. the men\n\nA. take a ladder to board the boat on their backs.\nB. climb across the rope.\nC. drives to a lake and they all go fishing.\nD. then jump down the side of a bridge.","prompt":"The men begin putting a boat on the back of a truck. The men tie the boat down. the men","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6445,"error":null,"output":"C"}
{"question_id":"hellaswag:00001330","item_index":159,"attempt":0,"prompt_hash":"e2c72667aedcec0484dd439eb4324631a3b0d7b192642f3b23813123ab548a58","question":"A man in yellow glasses takes a drink out of a water bottle. People are running across a finish line of a race. someone\n\nA. falls back on the sand and gets trampled.\nB. puts a medal around another person's neck.\nC. crashes into the backstretch lane.\nD. holds up crumpled sticky things.","prompt":"A man in yellow glasses takes a drink out of a water bottle. People are running across a finish line of a race. someone","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6443,"error":null,"output":"B"}
{"question_id":"hellaswag:00008142","item_index":160,"attempt":0,"prompt_hash":"607b13ce3fdf7959196b21f32825080c91f90921d723c8d36f02e3682ce68451","question":"[header] How to contact long island medium [title] Fill out the contact form on her website. [step] Theresa's website has a \" contact \" page, which is her preferred method for requesting a reading. The \" contact \" page has a form for you to fill out.\n\nA. To use this page, simply visit theresa's website and search for the medium you want to contact. [substeps] Theresa's website usually has an \" email contact \" section.\nB. The form asks for your name, address, phone number, zip code, and the following information : [substeps] Your personal contact information. The form includes email address and call to find your contact information.\nC. You can sign and date the entire body of the form. To use this form, enter the following information on the upper-right corner of the page : [substeps] Your name and contact information as well as your country code (e.g.\nD. You should fill it out as completely as possible, with your name, email address, city, and phone number. You should also specify whether you're looking for a private or group reading.","prompt":"[header] How to contact long island medium [title] Fill out the contact form on her website. [step] Theresa's website has a \" contact \" page, which is her preferred method for requesting a reading. The \" contact \" page has a form for you to fill out.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6806,"error":null,"output":"D"}
{"question_id":"hellaswag:00006479","item_index":161,"attempt":0,"prompt_hash":"1a85a9423ebd17690767877de6f6b446b5accc8a7dd6ef2f9e61cf2c325c0979","question":"[header] How to get vaseline out of your hair [title] Blot your hair with paper towels. [step] Take a few paper towels and press them on the hair that's coated with vaseline. Dab and blot with the paper towels to remove the excess vaseline.\n\nA. [substeps] Ensure that you don't rub the paper towels which would work the petroleum deeper into your hair. If you're trying to get vaseline out of your own hair, you may want to ask a friend to help you blot the areas you can't see yourself such as the back of your head.\nB. If any vaseline remains on your hair from you wiping it off, you'll need to wait a few minutes for the excess to soak into your hair. [title] Mix a paper towel with warm water.\nC. Put on some more paper towels and dab a little more vaseline over the areas that have vaseline on them. For the fastest results, dab the vaseline until it has soaked into your hair.\nD. As you keep blotting, get creative. [substeps] If you don't have paper towels, any cloth or towel will do.","prompt":"[header] How to get vaseline out of your hair [title] Blot your hair with paper towels. [step] Take a few paper towels and press them on the hair that's coated with vaseline. Dab and blot with the paper towels to remove the excess vaseline.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6806,"error":null,"output":"A"}
{"question_id":"hellaswag:00009211","item_index":162,"attempt":0,"prompt_hash":"753f171b765ddb0da7d9a6b03574df2d3900304cf0af4d0ef4235a99463558b7","question":"[header] How to play the knife and fork chocolate game [title] Place the bar of chocolate in the middle of the table. [step] The chocolate should be kept in its original foil and paper wrapper. If your chocolate is already unwrapped, wrap multiple layers of wrapping paper around it.\n\nA. [substeps] Use caution when using foil wrapped around chocolate; it could break a wire \" cooling \" mechanism. For any color chocolate, place a carrot side on the paper wrapping paper.\nB. Poke a hole in one edge so the chocolate sticks out when played. [title] Flip the block of chocolate (the one that is at the center of the dinner table) so that there is a gold bar underneath it.\nC. [title] Gently lay the cut bar of chocolate upside down on the edges of the table. [step] You should put the foil away from the chocolate as you would with an ordinary chocolate bar.\nD. [substeps] If you don't have wrapping paper handy, try wrapping with newspaper. Though not as pretty, this simple fix will up the difficulty of the game.","prompt":"[header] How to play the knife and fork chocolate game [title] Place the bar of chocolate in the middle of the table. [step] The chocolate should be kept in its original foil and paper wrapper. If your chocolate is already unwrapped, wrap multiple layers of wrapping paper around it.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6806,"error":null,"output":"C"}
{"question_id":"hellaswag:00004625","item_index":163,"attempt":0,"prompt_hash":"c43135f9e924d0f6ff960e10b1aadc1a23f548d2ce106163f035361750cb9e13","question":"[header] How to take off clothes in a sexy way [title] Prepare in advance (optional). [step] You can use these tips any time you're taking off clothes, no matter what you're wearing and how far you intend to go. But if you want to give your sexual partner a special striptease, choose your clothing and location in advance.\n\nA. Wear lacy or semi-transparent undergarments, stockings with garters, and a couple layers of sexy, easy-to-remove clothing over this. Consider placing one chair for your partner to sit in, and one for yourself to use during the performance.\nB. You don't want to waste a successful and exciting night with your only underwear on while you're making sure all the panty lines are in place. On the other hand, don't rush out to your destination with no panties on.\nC. Choose your lingerie, sheets, and other items to consider before leaving the house after you take them off. [substeps] If you're leaving the house to get dressed and you have trouble finding the venue, ask for help.\nD. Feel free to wear any one of the following well-defined clothes, to indicate when you plan to go. If you know your best place to get naked, consider a simple bra or strapless dress that you can wear with your panties for a sexy outfit.","prompt":"[header] How to take off clothes in a sexy way [title] Prepare in advance (optional). [step] You can use these tips any time you're taking off clothes, no matter what you're wearing and how far you intend to go. But if you want to give your sexual partner a special striptease, choose your clothing and location in advance.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6806,"error":null,"output":"A"}
{"question_id":"hellaswag:00003673","item_index":164,"attempt":0,"prompt_hash":"28adceb9430d42c453bfdba89a9a5ffa1e43cf194c4b68fff93a4818503232c2","question":"[header] How to thread your eyebrows [title] Get sewing thread. [step] To thread your eyebrows, you will need high-cotton sewing thread. Measure out a piece of thread that is as long as your forearm, plus three inches.\n\nA. Use this length to try and get straight stitches on your eyebrows. [substeps] To get started, grab 2 pieces of thread.\nB. If you feel that you can do it with your wrist, then do so. If you are not able to sewing by hand, then get some regular thread.\nC. It should be around 14 inches or 35 cm long. [substeps] Using high-cotton sewing thread will ensure your eyebrow hairs do not snap off between the thread.\nD. Tie this thread on the outermost thread ring. Push the needle to a place where it won't fall out of the ring.","prompt":"[header] How to thread your eyebrows [title] Get sewing thread. [step] To thread your eyebrows, you will need high-cotton sewing thread. Measure out a piece of thread that is as long as your forearm, plus three inches.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6808,"error":null,"output":"C"}
{"question_id":"hellaswag:00006319","item_index":165,"attempt":0,"prompt_hash":"8bea73bfc2ef2eb01d6f0c7405083be1752c22e6bd2ae925abbb4c8bb38bcf1a","question":"[header] How to install a hot tub [title] Check to see if you need a building permit. [step] Many cities will require building permits for installing hot tubs outdoors. Check with yours to see if you need to as well.\n\nA. Most public building permits only come with license agreements, but you could still get a construction permit. [substeps] If you are a new construction worker, inquire about other licenses, permits, and improvements you might need for a home.\nB. [title] Choose a location for your hot tub. [step] Make sure the spot you choose is large enough for the hot tub as well as some extra leeway for getting in and out and performing maintenance.\nC. [substeps] If you are planning to build a hot tub out of the construction projects that you decide to purchase, you'll need a building permit. This permits you to use the hot tub on public housing.\nD. You can get a red and blue permit by completing your assignment and then completing your application before adding enough money to purchase the hot tub. [title] Understand buying a hot tub under a permit is not required.","prompt":"[header] How to install a hot tub [title] Check to see if you need a building permit. [step] Many cities will require building permits for installing hot tubs outdoors. Check with yours to see if you need to as well.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6805,"error":null,"output":"B"}
{"question_id":"hellaswag:00009039","item_index":166,"attempt":0,"prompt_hash":"3d7e75317bceaa4a343c4ffd847ff0f870ebcfaec2e02f805467913a0aad23d9","question":"[header] How to get a haircut for curly hair [title] Choose a hair cut which will suit the shape of your face. [step] In addition to your curl pattern and your hair texture, various cuts will look different on you depending on the shape of your face. Pull your hair away from your face and take a self-portrait or trace the shape of your face on a mirror.\n\nA. [title] Search the internet to find hairstyles suited to your type. [step] Patterns can be hard for many to identify, and usually agencies will use pictures as reference during a fashion search.\nB. [substeps] The exact type of style you choose will depend on how curly you want your hair to be. Ask a hairstylist if a special will cut your hair for you or other people.\nC. [substeps] If your hair looks good, and hair has been styled so it fits your shape, wear it. If you prefer a more voluminous look, split the hair into two sections.\nD. [substeps] If your forehead is the widest part of your face, you likely have an oval face. If your cheeks are the widest part of your face, you may have a round face shape.","prompt":"[header] How to get a haircut for curly hair [title] Choose a hair cut which will suit the shape of your face. [step] In addition to your curl pattern and your hair texture, various cuts will look different on you depending on the shape of your face. Pull your hair away from your face and take a self-portrait or trace the shape of your face on a mirror.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6806,"error":null,"output":"D"}
{"question_id":"hellaswag:00006397","item_index":167,"attempt":0,"prompt_hash":"66e1af7ef20f31c7067e9b796222c2f0f46e6603dd0d0ebbbdb0028ca3663b72","question":"[header] How to style baby hair [title] Spray your baby hairs with water. [step] It's easiest to style your baby hairs when they're clean and free of product. You don't need to soak you whole head, just the baby hairs.\n\nA. Leave your baby hairs to sit in the shower for several minutes so they have some time to absorb shampoo and conditioner. [substeps] Avoid the urge to do the entire session with your hand.\nB. You can use a spray bottle full of water to spray along your hairline. [substeps] If you don't have a spray bottle, just splash water onto your hairline.\nC. Your baby hairs need to be completely wet, as they'll smooch. [substeps] Run water over your baby hairs to remove as much sebum as possible.\nD. Just spray them until your baby hairs are dry, rather than holding them in water. [substeps] To clean your baby hairs with water, fill a spray bottle or strainer with warm water and open the cap.","prompt":"[header] How to style baby hair [title] Spray your baby hairs with water. [step] It's easiest to style your baby hairs when they're clean and free of product. You don't need to soak you whole head, just the baby hairs.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6806,"error":null,"output":"B"}
{"question_id":"hellaswag:00008919","item_index":168,"attempt":0,"prompt_hash":"68045c44258ac7f70a429e25f877bdfbac31dacba2e6777d7b60c5a8ec58dca8","question":"[header] How to choose a pet rabbit [title] Visit the breeder that you are getting the rabbit from. [step] Look at the location the rabbits are in and check that it's clean, has water and food, and seems neat and pleasant. [title] Look at the rabbits.\n\nA. [step] Are they very smelly and happy? Are they healthy? A healthy rabbit will have shiny hair and wide eyes, and like to be groomed, then they are likely to be a good choice. A healthy rabbit who eats healthy food will be bright red with dark fur and is energetic.\nB. [step] There are four different types of rabbits-white and black. White rabbits are known to live longer than black rabbits do.\nC. [step] Find a few you like the appearance of and ask the breeder to separate these from the rest of the bunnies so you can choose one. [title] Note that you should bond with the bunny before you make you final pick.\nD. [step] Start by asking whether they are male or female. Males are generally older (around 40-55), females about 40-65.","prompt":"[header] How to choose a pet rabbit [title] Visit the breeder that you are getting the rabbit from. [step] Look at the location the rabbits are in and check that it's clean, has water and food, and seems neat and pleasant. [title] Look at the rabbits.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6806,"error":null,"output":"B"}
{"question_id":"hellaswag:00003913","item_index":169,"attempt":0,"prompt_hash":"08eda3dba368ace4088f96de0acaad4d553eaa0d5fb1bec1b0691d33b5745e9c","question":"[header] How to perform a no comply pressure flip on a skateboard [title] Get the feet correctly positioned. [step] Set up with your back foot off the tail of the board with just your big toe in the pocket. This is exactly the same position as the regular pressure flip.\n\nA. Your feet should be perpendicular with the board, but farther apart. [title] Lift the toes of your board up.\nB. This trick is almost all in the big toe of back foot, so this step is pretty important. [substeps] Your front foot can be wherever it is comfortable, perhaps just under the bolts at an angle.\nC. [substeps] Make sure your ankle can't be crossed over that foot. You'll keep your hand gripping the tail.\nD. You may need a little more force to get the toe where it is. You'll know when to let go of the tail.","prompt":"[header] How to perform a no comply pressure flip on a skateboard [title] Get the feet correctly positioned. [step] Set up with your back foot off the tail of the board with just your big toe in the pocket. This is exactly the same position as the regular pressure flip.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6807,"error":null,"output":"B"}
{"question_id":"hellaswag:00005516","item_index":170,"attempt":0,"prompt_hash":"f9d5c0ef18476bb0ffba9295913991bd29f778ca8a873cd95025b8ffe66d13a6","question":"[header] How to be a fashion designer [title] Take relevant high school classes. [step] If you're considering a career in fashion design, it's never to early to start exploring your interests and gaining experience. You can even begin your fashion design education in high school, taking relevant coursework that can help you get a head start on your college education.\n\nA. You can get technical and technical class opportunities anytime you want. [substeps] Join a crafting group to help you get to know more.\nB. [substeps] Study as many fashion design principles as you can. Become an international designer if you can in your education.\nC. [substeps] It's important to get-up start as soon as possible as getting a degree in fashion is one of the more important aspects of that industry. The fact of the matter is, anyone who succeeds all the time has a reason to help a fashion designer gain popularity.\nD. [substeps] Because designs often start as drawings/sketches, it would be useful to take any art classes your school offers. Many high schools now offer computer-aided design (cad) classes.","prompt":"[header] How to be a fashion designer [title] Take relevant high school classes. [step] If you're considering a career in fashion design, it's never to early to start exploring your interests and gaining experience. You can even begin your fashion design education in high school, taking relevant coursework that can help you get a head start on your college education.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6807,"error":null,"output":"D"}
{"question_id":"hellaswag:00000631","item_index":171,"attempt":0,"prompt_hash":"1a463ecbd4b99a7897e5d95f33cacb7b07d0334fdd658993dd46f0623c5bc59a","question":"The chemical is placed at the sit of the wooden chair. The person scrubs the wooden chair with a brush. The person shows the brush bristles now browned. the person\n\nA. rinses the wooden chair with the garden hose.\nB. adjusts a light blue soap product on the item and rubs the item.\nC. shines the metal handle of the metal polish brush on the shoe from the two round item placed on top of the chair.\nD. set the stool on the side of the chair to show the underside of the chair.","prompt":"The chemical is placed at the sit of the wooden chair. The person scrubs the wooden chair with a brush. The person shows the brush bristles now browned. the person","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6807,"error":null,"output":"A"}
{"question_id":"hellaswag:00002960","item_index":172,"attempt":0,"prompt_hash":"cd28ca0fce36abd08773161a51dc7690ab3c36d001c0d5574b5a54966d7386cb","question":"Inside of a fence men are sitting upon horses waiting. one of the men\n\nA. starts to chase after a bull on his horse.\nB. lays down and get tied.\nC. gets on the horse and begins to ride.\nD. pulled the string ladder onto his horse with the other and then led the horse on a tether rope.","prompt":"Inside of a fence men are sitting upon horses waiting. one of the men","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6807,"error":null,"output":"C"}
{"question_id":"hellaswag:00003399","item_index":173,"attempt":0,"prompt_hash":"b968460444ce82b7624f800702e9493f50847dfb7078af1b61b8c4f900d3ed55","question":"[header] How to coordinate a fashion show [title] Decide on a theme. [step] All fashion shows are centered around a general theme. [substeps] A theme could be fall fashion, beach fashion, lingerie, or costumes.\n\nA. Or it could be a secondary theme like a junior prom or new year's eve. [title] Tell the directors of your fashion show.\nB. Alternatively, the theme could focus on a particular color or fabric. The music, lighting, and decorations will have to go along with this theme.\nC. If you believe fashion is important to each audience, brainstorm topics of interest to your audience. A theme could include asian cuisine, outdoor skydiving, obedience classes, etc.\nD. Your choice could be a superhero themed fashion show, boys sports professional, or photography. [title] Pick april 1st or july 15th, which are theme days.","prompt":"[header] How to coordinate a fashion show [title] Decide on a theme. [step] All fashion shows are centered around a general theme. [substeps] A theme could be fall fashion, beach fashion, lingerie, or costumes.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6807,"error":null,"output":"B"}
{"question_id":"hellaswag:00003001","item_index":174,"attempt":0,"prompt_hash":"44f5cf8b6380cfb512b3a5638526c9f4984765b58391983450be2395e58b231c","question":"Two teams of basketball players are on a court. They throw the ball and engage in a game of dodgeball. they\n\nA. cheer as they win the game.\nB. run and try not to get hit by the ball.\nC. run around the court making plays and bumping into each other.\nD. run and hit the ball back and forth through the net.","prompt":"Two teams of basketball players are on a court. They throw the ball and engage in a game of dodgeball. they","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6805,"error":null,"output":"C"}
{"question_id":"hellaswag:00005401","item_index":175,"attempt":0,"prompt_hash":"27008ba7b306cb9c0e0651926c112bf2b24573b8782dae5156318f78bf3d2742","question":"[header] How to write business emails [title] Summarize the email in a 6-8 word subject. [step] It is important to make sure you include a subject, since an email that does not have a subject may be ignored. Your subject should be short and to the point and it should highlight the main message of the email.\n\nA. [substeps] If you are writing to a business partner or a friend, try to summarize these first sentences in a 1-2 paragraph citation. The body of the citation should tell them just what you're sending and why.\nB. [substeps] If your email is rhetorical, mention that on the subject line. [title] Include a signature at the bottom of the email and leave extra space at the bottom.\nC. The ideal subject gives the reader all they need to know or informs them they need to make a decision. People who receive hundreds of emails a day may only open emails with particularly compelling subjects.\nD. [substeps] For example, if the subject of a business email is \" your tomato...\" then it should highlight it in the sender email field. If it is a marketing email, then it should highlight it in the sender email field.","prompt":"[header] How to write business emails [title] Summarize the email in a 6-8 word subject. [step] It is important to make sure you include a subject, since an email that does not have a subject may be ignored. Your subject should be short and to the point and it should highlight the main message of the email.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6806,"error":null,"output":"C"}
{"question_id":"hellaswag:00008518","item_index":176,"attempt":0,"prompt_hash":"83594c52831d08469a1d17798e38834bcec3eda4a37187c9c8955a91264aa793","question":"[header] How to use hair toner [title] Wait until your hair is at the right shade to tone. [step] You can't tone whenever you want. To achieve the color tone you want, you need your hair to be at the right shade of yellow.\n\nA. This can be achieved, but it takes a significant amount of time. [substeps] Waiting until you've reached your desired shade is when your hair is at your ideal shade.\nB. Keep this in mind when selecting hair toners. [substeps] Use a neutrally colored toner to tone your hair.\nC. If you want a light ashy or cool color, you have to make sure your hair is at a light yellow color before using the toner. [substeps] If you use toner at the wrong shade of yellow, you won't get the result you want.\nD. [substeps] Check the underside of your hair once a week for light yellow areas. This is the area on your scalp that gets the most golden brown.","prompt":"[header] How to use hair toner [title] Wait until your hair is at the right shade to tone. [step] You can't tone whenever you want. To achieve the color tone you want, you need your hair to be at the right shade of yellow.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6895,"error":null,"output":"C"}
{"question_id":"hellaswag:00007572","item_index":177,"attempt":0,"prompt_hash":"9d0639baca17147a7938bc4cd0b6e877b3820914bec75704b6da1e2db36030db","question":"[header] How to cure oily skin [title] Use a gentle facial cleanser twice a day. [step] Most dermatologists agree that cleansing the skin is the best way to reduce oil. Make sure to use a gentle, ph balanced cleanser rather than regular soap.\n\nA. This will help cut down on the amount of oil you use at every time you take a shower. It will also reduce the likelihood that pores will form, which will reduce your problem.\nB. Most soaps are highly alkaline and strip the natural acid mantle from the skin leaving it vulnerable to bacteria. [substeps] Products containing benzoyl peroxide, salicylic acid, glycolic acid, or beta-hydroxy acid are often considered best.\nC. [substeps] Often, commercial cleansers are advertised as soft and ph clean, which is not good for oily skin. If you take the time to soak in a bubble bath, you can feel the bubbles from the water moving under your skin as you gently exfoliate.\nD. [substeps] Wash your face once a day with warm water. Lather up and let the cleanser sit in the skin for 15 to 30 minutes.","prompt":"[header] How to cure oily skin [title] Use a gentle facial cleanser twice a day. [step] Most dermatologists agree that cleansing the skin is the best way to reduce oil. Make sure to use a gentle, ph balanced cleanser rather than regular soap.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6894,"error":null,"output":"B"}
{"question_id":"hellaswag:00000587","item_index":178,"attempt":0,"prompt_hash":"b759af8272fbbf074ebc1ad60813b11cb6e800e29b0e9fe30e6c0ad8c17aefb2","question":"A man mows his lawn while being filmed from a high angle in a sped-up video. the man\n\nA. cuts fence paper using his hands.\nB. follows various landmarks as he mows the grass as well as spectators.\nC. walks across the mowed lawn and sits on a bench.\nD. is now outside shown standing in various locations and his mower slowing down as he goes in and out of frame.","prompt":"A man mows his lawn while being filmed from a high angle in a sped-up video. the man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6897,"error":null,"output":"C"}
{"question_id":"hellaswag:00008659","item_index":179,"attempt":0,"prompt_hash":"3f067466efcbb8a587bcc680910c14cb9afa77385b41e8b8b770e33d71daad26","question":"[header] How to know if the guy you're dating is right for you [title] Think about your beliefs and values. [step] If the person you're dating is right for you, the two of you will share common core beliefs and values. While these values will vary from couple to couple, it is important to think about what is important to you in life and whether or not your partner also values these things.\n\nA. [substeps] Religion takes a backseat in picking a partner. It's better to remain informed about your beliefs and family traditions, rather than spying on people who may not be certified to be of that faith.\nB. [substeps] Is it a decision on your part? If so, then don't take it lightly. In the end, it's all your own control.\nC. [substeps] It is important to consider this as many times a day as possible in your relationship. You and your partner may make better friends this way.\nD. [substeps] Beliefs and values are not the same as common interests. For example, love, care, trust, and respect are values, while sports and television are interests.","prompt":"[header] How to know if the guy you're dating is right for you [title] Think about your beliefs and values. [step] If the person you're dating is right for you, the two of you will share common core beliefs and values. While these values will vary from couple to couple, it is important to think about what is important to you in life and whether or not your partner also values these things.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6895,"error":null,"output":"D"}
{"question_id":"hellaswag:00009057","item_index":180,"attempt":0,"prompt_hash":"ceabb1c858201bd833e2a376bdb4118aaa043d83da61500deace52628a9f09ef","question":"[header] How to prepare skin before makeup [title] Purchase a facial cleanser that is right for your skin. [step] The cleanser should be alcohol-free and gentle. Though many products on the market offer miraculous results, they are often just hype.\n\nA. Look for cleaners that have simple ingredients that won't damage your skin. [title] Choose a ph-balance that meets your needs.\nB. Generally, people with oily skin will have a' sweet' skin, however, gentle products can still help take the place of the harsher scrubs. Ideally, you should use a facial cleanser with : [substeps] Benzoyl peroxide or a cleanser with beta hydroxy acids.\nC. Read your label to find out exactly what products are alcohol-free. Talk to your doctor or dermatologist to purchase a facial cleanser that contains alcohol.\nD. Nor are they gentle enough to treat facial blemishes and acne. [substeps] While it might seem like it would be flattering to put a natural, unscented cleanser on your face, skipping the shower or a brush to scrub face is not as helpful.","prompt":"[header] How to prepare skin before makeup [title] Purchase a facial cleanser that is right for your skin. [step] The cleanser should be alcohol-free and gentle. Though many products on the market offer miraculous results, they are often just hype.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6895,"error":null,"output":"C"}
{"question_id":"hellaswag:00005853","item_index":181,"attempt":0,"prompt_hash":"4547318a2b8f57a0454c24fb312ab8da285d8035dc9c3710905d50bc60d49fde","question":"[header] How to get rid of permanent coontails in your hair [title] Buy a high-quality bleach that can get rid of the darkest color in your coontails. [step] Some good products include manic panic. [title] Prepare half of the bleach and set the rest aside.\n\nA. [step] Mix 1 cup (240 ml) of water with 1 pint (0.95 l) of baking soda until it dissolves. [substeps] Make sure you use the sugar, chlorine and baking soda solution.\nB. [step] Go to the' suggestion section' of the beauty aisle of any beauty supply store and buy the amount needed from 20% , 250 milliliters. Put the rest in a deep container with 2 to 5% bleach in it.\nC. [step] There will be more bleach but you can probably get away with a few bottles. [title] Polish the coontail needle by squeezing it firmly in.\nD. [step] You'll use the rest of it in a little bit, so don't throw it away. [title] Separate the coontail from the rest of your hair.","prompt":"[header] How to get rid of permanent coontails in your hair [title] Buy a high-quality bleach that can get rid of the darkest color in your coontails. [step] Some good products include manic panic. [title] Prepare half of the bleach and set the rest aside.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6895,"error":null,"output":"D"}
{"question_id":"hellaswag:00008891","item_index":182,"attempt":0,"prompt_hash":"d811610040e5d232e2be80f2b116afd264d39c3856652ed8206720e3730d8a31","question":"[header] How to diagnose stress response syndromes [title] Identify triggers of stress response syndrome. [step] Many events and situations can trigger a stress response. A major life stressor can include living through a natural disaster (like a flood, hurricane, or earthquake), ending a relationship, starting a new school or job, or the death of a loved one.\n\nA. A stress response can also occur due to multiple events such as marital problems or school problems. Adjustment problems can occur from recurrent events (like going to chemotherapy or driving in a car after an accident) or continuous events, such as living in an unsafe neighborhood.\nB. You can potentially trigger stress response syndrome by experiencing significant disruptions to life's daily routine. [substeps] Triggers are things you do as a teenager or teen and tend to experience during a teenage years.\nC. [substeps] On the contrary, whether the stress response is triggered by a traumatic event or a death of a loved one, it can impact other conditions of your life. This allows you to better address the emotional pain that is triggered by stress response syndrome.\nD. Stress response disorder in teenagers is often someone else's or a personality trait, and the cause is unknown. No severity is known, but the teen may experience the condition in addition to exhibiting symptoms.","prompt":"[header] How to diagnose stress response syndromes [title] Identify triggers of stress response syndrome. [step] Many events and situations can trigger a stress response. A major life stressor can include living through a natural disaster (like a flood, hurricane, or earthquake), ending a relationship, starting a new school or job, or the death of a loved one.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6895,"error":null,"output":"A"}
{"question_id":"hellaswag:00000473","item_index":183,"attempt":0,"prompt_hash":"bbe3000db00a9008832826ed1f65eb5ee5c6c06b2615fa98fb313df37ffc6b4a","question":"A man is wearing full running gear. He laces up his shoes and joins in a race. the people\n\nA. continue on running around and neon signs flash.\nB. are really excited about it as they race and jump high over low hills.\nC. line up, then run, racing each other to the finish line.\nD. form a giant circle and begin to race each other.","prompt":"A man is wearing full running gear. He laces up his shoes and joins in a race. the people","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6895,"error":null,"output":"C"}
{"question_id":"hellaswag:00007512","item_index":184,"attempt":0,"prompt_hash":"4977434d967647247c28eea66374b520ba7a5cb070d2760bc59ab31111866621","question":"[header] How to get healthy looking nails [title] Trim your nails regularly. [step] Use manicure clippers to trim your nails straight across and round the tips with a gentle curve. It's easier to trim your nails after a bath or shower since they are soft and more gentle to clip.\n\nA. Don't file your nails immediately after exposure to water as this may weaken your nails. [title] Buff your nails for a healthy shine.\nB. Only clip down on the length of each nail. Don't trim straight across from the tip of your first one; you risk cutting yourself on this.\nC. [substeps] While you're in the shower, trim back the length of your nails according to your preference. This will give you an extra length for later.\nD. It may also work to file your nails so your nails are less curved. [substeps] To prevent staining your nails, add ice packs to help you keep the ice pack cool.","prompt":"[header] How to get healthy looking nails [title] Trim your nails regularly. [step] Use manicure clippers to trim your nails straight across and round the tips with a gentle curve. It's easier to trim your nails after a bath or shower since they are soft and more gentle to clip.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6895,"error":null,"output":"A"}
{"question_id":"hellaswag:00007422","item_index":185,"attempt":0,"prompt_hash":"2232fe4ec620b45761a0f4099cf3155c212fbb4c0ab584fe989cf44bafcad58d","question":"[header] How to make an adult blanket sleeper [title] Create your blanket sleeper's pattern. [step] Lay several feet worth of untouched christmas wrapping paper down on a large flat surface. Grab a pair of your night pants and a heavy long-sleeve sweatshirt on a flat surface so that both touch each other at the waistline.\n\nA. ( make this seem natural when placed down on the surface with the paper). Make sure the pieces are folded and laid in the same direction on the same side of the paper that should be folded over so that the white side is showing outward.\nB. Cut or tear any fabric and lay it flat to form the pattern. [substeps] Throw away unwanted clothing and fabric after sewing the pattern onto your pillowcase.\nC. Tear off a long strip of christmas wrapping paper and lay it under your blanket. Fold the paper down and tape it down to the pattern and next to your blanket.\nD. Fold each remaining portion of paper in half to form the long sleeve front fabric. Make sure to double-check the pattern to make sure that it is not jagged with sharp edges.","prompt":"[header] How to make an adult blanket sleeper [title] Create your blanket sleeper's pattern. [step] Lay several feet worth of untouched christmas wrapping paper down on a large flat surface. Grab a pair of your night pants and a heavy long-sleeve sweatshirt on a flat surface so that both touch each other at the waistline.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6895,"error":null,"output":"C"}
{"question_id":"hellaswag:00003861","item_index":186,"attempt":0,"prompt_hash":"73813ce3ac59a5ce913c0929a389cb8cca50d104eaf5690151e788072cd73055","question":"[header] How to clean dr brown bottles [title] Dissemble the bottle. [step] To effectively clean any bottle, it's important to take it apart. However, dr. brown's bottles have more parts than traditional baby bottles, so you must be sure to fully disassemble it.\n\nA. Make a show of all the parts inside, including the bottom and the back sides of the bottle. [substeps] You can also wear gloves and an apron to get access to the inside.\nB. Then, use a screwdriver to loosen the screw on the screw on the inside. Gently pull the bottle apart so that the screw and screw head meet.\nC. Use a sharp knife to carefully remove the bottle. [substeps] Remove as much air as possible from the bottle, even if its in the refrigerator.\nD. Separate the cap, nipple collar, nipple, vent insert, vent reservoir, travel disc, and any insert components from the bottle so you can clean each part. [substeps] If you're not sure how to take apart your particular dr.","prompt":"[header] How to clean dr brown bottles [title] Dissemble the bottle. [step] To effectively clean any bottle, it's important to take it apart. However, dr. brown's bottles have more parts than traditional baby bottles, so you must be sure to fully disassemble it.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6894,"error":null,"output":"D"}
{"question_id":"hellaswag:00008767","item_index":187,"attempt":0,"prompt_hash":"615d2ba6a653b823a37933b2644f3b0c9dbfd8c2a4560634daa0d50854896d37","question":"[header] How to use your fitbit to monitor your sleep [title] Wear the fitbit flex tracker before you hit the sack. [step] Strap it around your arm firmly but not too tight. [title] Put the flex into sleep mode.\n\nA. [step] Tap your fitbit flex rapidly for at least two seconds to enter its sleep mode. You'll see two alternating, blinking lights moving back and forth.\nB. [step] It goes into sleep mode on most day of the week. Make sure that you wear the tracker for at least 3 hours before you hit the sack.\nC. [title] Choose the appropriate app (fitbit flex main) to track you, such as \" fitbit fitbit xp k \". [step] For most such apps, you have to use a virtual recorder with a 500 meter screen.\nD. [step] The program will turn off if you lock it. [title] Once you sleep you might have to switch the tracker on again.","prompt":"[header] How to use your fitbit to monitor your sleep [title] Wear the fitbit flex tracker before you hit the sack. [step] Strap it around your arm firmly but not too tight. [title] Put the flex into sleep mode.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6527,"error":null,"output":"B"}
{"question_id":"hellaswag:00005498","item_index":188,"attempt":0,"prompt_hash":"7849052726b5860a8bea9b0120d1a43cde1768b25c63ee37881d74b5ed33d1bb","question":"[header] How to apply draping blush [title] Use the right type of brush. [step] The kind of brush depends on where you are going to apply the blush. Since you will be applying blush to more than just the apples of your cheeks, you will want to use a smaller brush for smaller areas.\n\nA. For the cheeks, you can use a regular blush brush. [substeps] Some suggest using a setting brush for the smaller areas of your face.\nB. The smaller brush you use, the bigger the blush you will need to add to your cheeks. Check the packaging for the correct brush size for this area.\nC. [title] Pick the brush that works best for your hair type. [step] For people with curly hair, use a larger brush for thinner hair and a smaller brush for thin hair.\nD. You can use a tube brush for larger areas, or a brush of a different color to avoid clashing. [substeps] Use a brush that has brush bristles that are the same colors as your foundation so that the rounded bristles appear less noticeable.","prompt":"[header] How to apply draping blush [title] Use the right type of brush. [step] The kind of brush depends on where you are going to apply the blush. Since you will be applying blush to more than just the apples of your cheeks, you will want to use a smaller brush for smaller areas.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6528,"error":null,"output":"C"}
{"question_id":"hellaswag:00002607","item_index":189,"attempt":0,"prompt_hash":"b7085e6ad3b066b4c69eb7b9f69ed8dfd18b81809cbf8e9e7966fe816d27f76f","question":"An african american female takes off running and begins to attempt a high jump but fails when the bar drops and walks off. another female\n\nA. shows up and tries her third attempt and the same thing happens.\nB. runs after her and baseball bat hits her in the head knocking her to the ground, and she lays there for a while trying to get up but the crowd once again drags her away.\nC. appears holding a small boy in her arms who is jumping up and down while she tries several different positions.\nD. joins her and she tries several more times then walks off score.","prompt":"An african american female takes off running and begins to attempt a high jump but fails when the bar drops and walks off. another female","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6527,"error":null,"output":"A"}
{"question_id":"hellaswag:00001735","item_index":190,"attempt":0,"prompt_hash":"fcbb2faece260249dc48a398ddb9693a1670013adc7beff542e6445c486b395c","question":"A man is dressed in a spiderman outfit. he\n\nA. is leaning against a rail inside a building.\nB. is standing on a roof of a home.\nC. is playing the drums while he talks.\nD. is using a needle to thread through shoelaces.","prompt":"A man is dressed in a spiderman outfit. he","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6527,"error":null,"output":"B"}
{"question_id":"hellaswag:00002595","item_index":191,"attempt":0,"prompt_hash":"0fbef12975704cc56c8c5ac8ce90acfa8ae16b5a646a14184333f60e415733a1","question":"Two men are in a room and the man with a blue shirt takes out a bench stone and with a little lubricant on the stone takes an knife and explains how to sharpen it. then he\n\nA. stops sharpening the knife and takes out some pieces of paper to show how sharp the knife is as he cuts slivers of paper with the knife.\nB. shows how to cut the bottom with the knife and place a tube on the inner and corner.\nC. uses a sharpener to smooth out the stone using the knife.\nD. bends down and grabs the knife and remove the appliance.","prompt":"Two men are in a room and the man with a blue shirt takes out a bench stone and with a little lubricant on the stone takes an knife and explains how to sharpen it. then he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6527,"error":null,"output":"A"}
{"question_id":"hellaswag:00009606","item_index":192,"attempt":0,"prompt_hash":"b76b60d43e90c55e0b4ab38adcd9d8aa8011a1dc738032072a6d729eb73def13","question":"[header] How to plan your child's first playdate [title] Teach your child how to be a good host. [step] Your child will likely not understand the concepts of good hosting unless you teach them. Explain that good hosting helps guests feel welcome.\n\nA. Over time, this will be a great opportunity for your child to network and be comfortable living together. [substeps] Your child's goals and understanding of a house could help them put their minds at ease when the two of you go on your first date.\nB. Playdates are more fun when everyone feels like they are wanted there. [substeps] Good hosting begins with greeting the guest at the door.\nC. If your child has never been to a play, talk to your child about letting your child be active in a soccer game at a local playground. Encourage your child to be able to help others play it well.\nD. [substeps] Explain to your child their obligations before you introduce a new guest. This may involve teaching your child the purpose of help cooking or taking place under your house's roof.","prompt":"[header] How to plan your child's first playdate [title] Teach your child how to be a good host. [step] Your child will likely not understand the concepts of good hosting unless you teach them. Explain that good hosting helps guests feel welcome.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6305,"error":null,"output":"B"}
{"question_id":"hellaswag:00003504","item_index":193,"attempt":0,"prompt_hash":"f1f2815c1208b412e228d63b6be4edef94313064b0f97e3a312db2f5a30bebe9","question":"[header] How to make powdered sugar [title] Choose a blender or spice grinder. [step] Keep in mind that sugar can scratch plastic or weak glass. A high-powered blender is the fastest and most effective option, but any blender or spice grinder can do the job.\n\nA. The same old rule applies here: you can grind your sugar with the spice grinder but store it in a canning jar or tupperware with a lid. [substeps] If your sugar grinder doesn't come with one, consider using a hand blender instead.\nB. [title] Pour 1/4 cup of powdered sugar into the blender or spice grinder. [step] If you are using a heavy handle blender, hold the handle firmly enough to not strain the sugar.\nC. [substeps] Spice grinder or coffee grinders-which are essentially the same tool-pick up flavors from the ingredients they grind. Clean it thoroughly before using it for sugar.\nD. [substeps] A hand-held blender, like the dairy counter blender, should be used for grinding sugar (or rice and cornstarch). [title] Mix together two cups water and sugar.","prompt":"[header] How to make powdered sugar [title] Choose a blender or spice grinder. [step] Keep in mind that sugar can scratch plastic or weak glass. A high-powered blender is the fastest and most effective option, but any blender or spice grinder can do the job.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6304,"error":null,"output":"B"}
{"question_id":"hellaswag:00009886","item_index":194,"attempt":0,"prompt_hash":"47ab4db656d0950acfcc17d1a27f125fa92d83da3e388b6ccff5c10176cbbea2","question":"[header] How to make cake batter cinnamon rolls [title] Warm the milk. [step] Using a small saucepan, heat the milk under low heat. Have the milk heat up until warm, about 95 degrees fahrenheit or 35 degrees celsius.\n\nA. This should take about two to three minutes depending on your thermometer. [title] Pour the milk into a mixing bowl.\nB. It will usually take about 5-7 minutes. [title] Blend the milk, sugar, and yeast.\nC. Do not put the milk next to the cinnamon roll. [title] Add the dry ingredients and the butter to a medium mixing bowl.\nD. The resulting cake batter should look like a thick liquid, with no lumps or swirls. [substeps] Substitute butter for milk if you have none.","prompt":"[header] How to make cake batter cinnamon rolls [title] Warm the milk. [step] Using a small saucepan, heat the milk under low heat. Have the milk heat up until warm, about 95 degrees fahrenheit or 35 degrees celsius.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6308,"error":null,"output":"A"}
{"question_id":"hellaswag:00002113","item_index":195,"attempt":0,"prompt_hash":"94e8d2dcfb64473c019650a7e38147264891943bca68fba2266da849eaf89f25","question":"A woman is seen showing off a glass of honey. she\n\nA. slams a cup around on a table in front of her.\nB. mixes both ingredients in a glass and begins making rum with it.\nC. shows off the bottle of honey as well as a bowl of water and a towel.\nD. then puts a container into the glass and using a cloth to soak it up.","prompt":"A woman is seen showing off a glass of honey. she","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6307,"error":null,"output":"C"}
{"question_id":"hellaswag:00009205","item_index":196,"attempt":0,"prompt_hash":"9f2151da5036430aa97fb0cddc4148b0127831348fbf070c4a322c664ae1c9f9","question":"[header] How to teach your child not to bully [title] Set a positive example. [step] Demonstrate kind behavior when you're with your own friends, loved ones, and your children themselves. Children mimic the language and behavior of parents.\n\nA. If you want your child to be nice to other children, be nice to your child. [substeps] When your child is around, try your best to keep an even temper.\nB. [substeps] Do your best to be kind, but also listen to your loved ones when they are talking with other kids. If you sit down with your kids and speak in their presence, you will look removed from the conversation because you aren't there to explain things.\nC. For example, demonstrating good posture and being composed and assured allows your children to rely on you. [substeps] Don't be mean and use angry or disrespectful behaviors.\nD. Make it your goal to teach your child that being kind and accepting of all forms of childhood behavior creates more healthy self-esteem. [substeps] Do not try to force your kid to \" be nice \" by using words such as \" he doesn't want to be mean, \" or \" he wants to feel confident.","prompt":"[header] How to teach your child not to bully [title] Set a positive example. [step] Demonstrate kind behavior when you're with your own friends, loved ones, and your children themselves. Children mimic the language and behavior of parents.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6304,"error":null,"output":"A"}
{"question_id":"hellaswag:00003031","item_index":197,"attempt":0,"prompt_hash":"651805c6e26a58f22bfa266792dfe07e49b0dff2cc132e723ed7b4486f16e573","question":"The graphic \"ice cream eating competition\" appears on screen. Three people are eating ice cream out of cups with their fingers. the woman on the far end\n\nA. finishes hers first, followed by the other two.\nB. of the table licks her spoon while balancing it in her mouth.\nC. looks down from her tablet.\nD. of the ice cream cup is licking the spoon, which is passed along with both kids putting the spoon into their mouth.","prompt":"The graphic \"ice cream eating competition\" appears on screen. Three people are eating ice cream out of cups with their fingers. the woman on the far end","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6307,"error":null,"output":"A"}
{"question_id":"hellaswag:00009270","item_index":198,"attempt":0,"prompt_hash":"b1a9383e423b1927e0d2c49880475916755e98c3b97ee3ab289660fba250d504","question":"[header] How to use kaufman cards [title] Determine what the child's speech difficulties. [step] To begin working with the kaufman cards, you must know what kit level to use. That means understanding what vowels, consonants, and syllable shapes the child has already mastered.\n\nA. The kaufman speech praxis test for children is often used to determine a child's level. [substeps] It's best to have a speech therapist administer the test to the child to ensure that the results are accurate.\nB. [substeps] If the child struggles with identifying sounds, or has a hard time understanding terms under \" active voice, \" say \" active voice \" followed by a + or a b in between the letters. [title] Find out about the child's vocabulary.\nC. How you use the cards determines which steps you take when you order the cards online, in person, and in the store. [substeps] Some online cards may offer more basic food cues, with half of it understood and the other half almost exclusively included in the kit.\nD. [substeps] If the child cannot speak, ask questions like, \" what is it that is causing the straining? \" or \" what does this sound like? \" read through the cards aloud to see which words are used in speech. Note the correct pronunciation for the child's native languages, and add to the description as needed.","prompt":"[header] How to use kaufman cards [title] Determine what the child's speech difficulties. [step] To begin working with the kaufman cards, you must know what kit level to use. That means understanding what vowels, consonants, and syllable shapes the child has already mastered.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6306,"error":null,"output":"A"}
{"question_id":"hellaswag:00000072","item_index":199,"attempt":0,"prompt_hash":"455b35ab56b123758b964793b8339618febd3b6964bc491b813ef85fac958968","question":"Various pictures of food are shown followed by a person cutting up vegetables and mixing various ingredients into a pot. he\n\nA. shows how the vegetables are chopped for seasoning and then sprinkled into broth and shown with the water boiling.\nB. cooks up a meal and each are shown eating inside the pot.\nC. boils food into a bowl and then adds some part of a salad into the bowl.\nD. then cuts up a piece of bread, scoops out the insides, and fries up some meat.","prompt":"Various pictures of food are shown followed by a person cutting up vegetables and mixing various ingredients into a pot. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6305,"error":null,"output":"D"}
{"question_id":"hellaswag:00007659","item_index":200,"attempt":0,"prompt_hash":"dfad1ca238602c0cff74569a6adc182f90562598479b33e4f4fe90422c11f5bc","question":"[header] How to use olaplex [title] Find a stylist you trust at an olaplex-affiliated salon. [step] Olaplex is a beauty product that is only sold in select salons. Find a salon that you like and trust that carries olaplex and make an appointment for a treatment.\n\nA. [substeps] Both olaplex and marketable hair care kits are available online. The hair care kit from fitbit, along with a shampoo and conditioner that contains sallex solution, is a cheap and easy way to create a salon treatment.\nB. Pick a local salon and ask the stylist how they treat grooming and hair styles. At a salon, you will likely be able to choose conservative hairstyles and those with cropped or wavy hair.\nC. [substeps] You could try a temporary consultation or experiment with different stylists in one salon's establishment. [title] Open up olaplex back art.\nD. Since the treatment is incorporated into your existing color/styling appointment, it won't add very much extra time to your time at the salon. [substeps] You can use the olaplex online salon locator at https: // locator.","prompt":"[header] How to use olaplex [title] Find a stylist you trust at an olaplex-affiliated salon. [step] Olaplex is a beauty product that is only sold in select salons. Find a salon that you like and trust that carries olaplex and make an appointment for a treatment.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6304,"error":null,"output":"D"}
{"question_id":"hellaswag:00003982","item_index":201,"attempt":0,"prompt_hash":"c6de8bf29368237b24559562632dee063b230832503d82a9d3eff14205e23e5b","question":"[header] How to stop a budgie from biting [title] Keep a diary of when your budgie bites. [step] The first step to addressing biting is to keep track of when biting occurs. Budgies may bite for a variety of reasons.\n\nA. How to address the behavior depends on its root cause. [substeps] Write down each time your budgie bites you.\nB. You may notice that your budgie bites when it sees or hears something harmful, or because it is a bit startled by something happening outside. You can contact a vet or juvenile behavior trainer to tell them you notice biting.\nC. For example, there may have been an accident or a person has bitten them at different times. Try keeping a diary of the time that the bite occurs.\nD. For example, they may find food more enjoyable than they are consumed, and they will bite defensively. This will help you pinpoint which behavior is causing the biting and how to address it.","prompt":"[header] How to stop a budgie from biting [title] Keep a diary of when your budgie bites. [step] The first step to addressing biting is to keep track of when biting occurs. Budgies may bite for a variety of reasons.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6304,"error":null,"output":"A"}
{"question_id":"hellaswag:00001671","item_index":202,"attempt":0,"prompt_hash":"6044df25048e0797155481b8c7890187919ab58a720d825e3706d1467c508058","question":"A man is standing with a fishing pole. he\n\nA. is fishing through a hole in the ice.\nB. is talking as he takes a couple of fish out of his net.\nC. is talking as he hooks a fish with his other hand.\nD. bends over and grabs a large, black fish off the ground.","prompt":"A man is standing with a fishing pole. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6305,"error":null,"output":"A"}
{"question_id":"hellaswag:00006091","item_index":203,"attempt":0,"prompt_hash":"0b9c501d934fd5a360650b0bfd9e298f5329781afa1b32c1e3d6fd52b9d8348b","question":"[header] How to perform a carotid massage [title] Ask the patient to lie on their back. [step] For safety purposes, a csm should be performed first supine (flat on the back) and then sitting up with at least 5 minutes of rest in between. After performing the maneuver in both of these positions the patient should be observed for 10 minutes while in a supine position.\n\nA. [substeps] Check for a pulse before performing the maneuver. It should only take a few seconds for your pulse to respond.\nB. [substeps] You may also perform this procedure if the patient has a blood transfusion. [title] Order the patient's blood tests.\nC. [substeps] You may stand a minute later and sit across from a csm patient who is in a reclining position with their knees bent and feet on the ground. The patient should lie about 3 feet away from you.\nD. If you're in a medical office, you can ask the patient to lie on an examination bed. If you're performing the csm at someone's home, ask the patient to lie on a couch or bed.","prompt":"[header] How to perform a carotid massage [title] Ask the patient to lie on their back. [step] For safety purposes, a csm should be performed first supine (flat on the back) and then sitting up with at least 5 minutes of rest in between. After performing the maneuver in both of these positions the patient should be observed for 10 minutes while in a supine position.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6306,"error":null,"output":"D"}
{"question_id":"hellaswag:00009331","item_index":204,"attempt":0,"prompt_hash":"55ca19b4bc4d37cf746e1643ef9d161dddac6e647f7581c10f357dad30bacb9f","question":"[header] How to brush thick long hair [title] Brush your hair by the tips first. [step] Start from about halfway down your hair, and slowly brush the knots and tangles out. [title] Slowly bring your hairbrush up higher after each stroke.\n\nA. [step] Don't brush up too high or use too much pressure. The longer you brush your hair, the less force you use to straighten it, creating an afro or longer wavy hair.\nB. [step] This leaves your head less achy because it doesn't pull as much on the hair. [title] When your hair is almost all the way brushed out, you can go over all of it again to make sure all tangles are out.\nC. [step] This may take several passes in between strokes so be patient. [title] If you have time, add a fold between your brush and the rest of your hair to even out the length of your hair for less.\nD. [step] This will help both sexes shed these long, thick hairs. [title] Continue combing and brushing out the whole hair to remove excess hair.","prompt":"[header] How to brush thick long hair [title] Brush your hair by the tips first. [step] Start from about halfway down your hair, and slowly brush the knots and tangles out. [title] Slowly bring your hairbrush up higher after each stroke.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6305,"error":null,"output":"B"}
{"question_id":"hellaswag:00001716","item_index":205,"attempt":0,"prompt_hash":"3cd6328b270307efe24c48071e927b0d6ab420c25c05b51fbcdef4eac820ac81","question":"A group of people are seated around a dining table. A man is playing a harmonica for the group. they\n\nA. applaud and cheer as they play.\nB. pull up chairs next to him as he strums.\nC. sit and listen intently as he plays.\nD. celebrate as he continues to play the harmonica for them.","prompt":"A group of people are seated around a dining table. A man is playing a harmonica for the group. they","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6304,"error":null,"output":"D"}
{"question_id":"hellaswag:00000287","item_index":206,"attempt":0,"prompt_hash":"d6915010c2e2b08851e49838fd6faf12e6553dcbf059731a25545d5ac9b9c7d9","question":"Three teens run and a boy throws the basketball to the basket, then the second teen grabs the ball and pass the ball to two teens disputing to have the ball. Then, three more boys run to practice the same steps. a young man\n\nA. wearing a yellow t-shirt practice basketball on the other basketball goal.\nB. wearing black santa claus suit give the ball to one of the young teens who take it, while his mother pushes the round ball to the other direction.\nC. practice on a gym floor, while three other boys practice in gym gym.\nD. play basketball with a kid while facing backwards.","prompt":"Three teens run and a boy throws the basketball to the basket, then the second teen grabs the ball and pass the ball to two teens disputing to have the ball. Then, three more boys run to practice the same steps. a young man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6305,"error":null,"output":"A"}
{"question_id":"hellaswag:00002708","item_index":207,"attempt":0,"prompt_hash":"b3be08aa4a689720de53955b1c17a7926fe86c97944a0159b9cd1bd18bcd7bff","question":"The video leads into several clips of track runners jumping over a bar with their jump being shown again in slow motion. several more people\n\nA. race through different obstacles as well as jumping over steep obstacles and their jump is shown again later.\nB. are shown jumping over the bars in slow motion and ends with more clips of people jumping while jumping.\nC. are shown jumping over the bar with one hitting the bar and not jumping over.\nD. are seen jumping over the bars as well as beginning a slow race down the track, laughing with one another, and ending with the closing captions shown.","prompt":"The video leads into several clips of track runners jumping over a bar with their jump being shown again in slow motion. several more people","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6304,"error":null,"output":"C"}
{"question_id":"hellaswag:00004825","item_index":208,"attempt":0,"prompt_hash":"063f0ac4a52580b5813a2bf44341cb7f1620afa84968cf6529ca7dd74b982d69","question":"[header] How to decorate a dining table [title] Leave your table mostly bare. [step] Do not put plates or silverware on the table. Make sure there is enough space on the table for you to use for cooking, writing, reading, and other daily activities.\n\nA. [substeps] By allowing your dining table to be mostly free of decorations and clutter, you can use the space for more than just eating! You can use wax or another product with a glossy finish to keep your table gleaming and looking great. [title] Lay a tablecloth or runner in the center of the table.\nB. You should also refrain from putting unnecessary hooks in the decor, such as hooks for pans, plates, and utensils. By letting your table decorate, you reduce the risk of hooks getting smudged or worn.\nC. For example, instead of \" it's great when rain waves break tomorrow, \" you can write \" rain lowers the temperature around our house on the weekends. \" in addition, it's okay to avoid putting your silverware/plates, pots, and pans on your table.\nD. Place foods left at the table on the floor, to sit on, or to place non-dominant guests on opposite corners of the room. This will make it more inviting for them.","prompt":"[header] How to decorate a dining table [title] Leave your table mostly bare. [step] Do not put plates or silverware on the table. Make sure there is enough space on the table for you to use for cooking, writing, reading, and other daily activities.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6143,"error":null,"output":"A"}
{"question_id":"hellaswag:00009197","item_index":209,"attempt":0,"prompt_hash":"d65c4ad661230697c25bb2ae28153ca6d3bfd4518a23e86020be4a8194cb436f","question":"[header] How to wear a scarf [title] Wrap it in a simple loop. [step] Of one of the most basic and attractive ways of wearing a scarf is just wrapping it in a loop around your neck. Wrap the scarf around your neck to create a single loop, with the tail ends loosely draped in the front.\n\nA. This will make it look like a scarf is wrapped around a particularly desirable part of your body. [substeps] Tie the loop into a simple knot so that the loop, with the tail ends, can be seen and stood out.\nB. Pull on the loop a bit to loosen it up and create a more casual appearance. If you're trying to beat the cold, you can wrap the scarf multiple times around your neck.\nC. You can hook the tail end over your shoulder or tie a knot at the end. [substeps] Never wrap the scarf around your neck.\nD. Then, round the loop at the end to create smaller loops that connect more naturally to the scarf. Keep the loop to your neck for a more casual look.","prompt":"[header] How to wear a scarf [title] Wrap it in a simple loop. [step] Of one of the most basic and attractive ways of wearing a scarf is just wrapping it in a loop around your neck. Wrap the scarf around your neck to create a single loop, with the tail ends loosely draped in the front.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6146,"error":null,"output":"B"}
{"question_id":"hellaswag:00008935","item_index":210,"attempt":0,"prompt_hash":"7ee2b938c2cce2475f8ed9ffafc1b1460b0d0b53f3d4777077d964d1b5cee72c","question":"[header] How to be adventurous [title] Let go of what is holding you back. [step] Inhibitions are feelings that make you self-conscious and unable to act in a relaxed and natural way. Perhaps you are shy, inexperienced or fearful.\n\nA. Acting inappropriately might loosen up those around you. Try to allow yourself to bounce back from your shyness.\nB. In order to let go of what is holding you back you must feel safe with yourself, your surroundings and the people around you. [substeps] Ask yourself why you want to be adventurous.\nC. When confronted by a serious problem, find ways to combat it by seeking the assistance of someone who can guide you through it. This person could be a trusted friend, a family member, or a holy war instructor.\nD. Letting go of these feelings sends the message to your brain that you are not a wild animal and that as a result, you are less inhibited. Remind yourself what you are more confident about: your body.","prompt":"[header] How to be adventurous [title] Let go of what is holding you back. [step] Inhibitions are feelings that make you self-conscious and unable to act in a relaxed and natural way. Perhaps you are shy, inexperienced or fearful.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6143,"error":null,"output":"B"}
{"question_id":"hellaswag:00002428","item_index":211,"attempt":0,"prompt_hash":"2ce5348384cadb95509c0fc60fca7a3616fdd117db7015de3aad784a36e9ab55","question":"A boy kicks a soccer ball in the yard while his father works. the adult\n\nA. mows the lawn with his child.\nB. ducks over to make room for the soccer ball in his hand.\nC. lets the boy go and gives him a taste of the game.\nD. tries to get the soccer ball out of the boy's reach.","prompt":"A boy kicks a soccer ball in the yard while his father works. the adult","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6144,"error":null,"output":"D"}
{"question_id":"hellaswag:00002197","item_index":212,"attempt":0,"prompt_hash":"ef947f75771bc43e12f8570e23be13446c0cd672a59b7e9ed975805b70fd84c4","question":"The man and woman continue talking to each other. It flashes back to the pool table. the man and woman\n\nA. continue talking and the woman laughs at what the man is saying.\nB. laugh about capoeira and the lady tosses the ball.\nC. joke with one another on the table.\nD. are standing with the help of referee.","prompt":"The man and woman continue talking to each other. It flashes back to the pool table. the man and woman","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6144,"error":null,"output":"A"}
{"question_id":"hellaswag:00008921","item_index":213,"attempt":0,"prompt_hash":"4071d3dedd24c3f5ea487060669dbbc42d99bd10888be94d7a26df3d41bf9f53","question":"[header] How to make your lips smooth [title] Drink 8 glasses (1.9 liters) of water daily. [step] Staying hydrated is a great way to keep your lips moist, smooth, and wrinkle-free. Drinking water also has many other benefits, including clear skin.\n\nA. For smoother, kissable lips, you should drink 8 glasses (1.9 liters) of water every day. [substeps] If you have a hard time staying hydrated you can try sparkling water, which is very naturally moisturizing and antioxidant-free.\nB. [substeps] If you notice your lips are looking more chapped than usual, drink some water. There are other ways to stay hydrated as well.\nC. [substeps] Drinking water completely removes plaque and dead skin cells from your lips, so it is much less likely to form oil. Avoid over-drinking alcohol and caffeine.\nD. [substeps] You can drink 8 glasses (1.9 liters) of water every day. One study found that people who drank 12 glasses (3.9 liters) of water a day kept their lips smooth and blemish-free and enhanced by their lip colour and lipstick.","prompt":"[header] How to make your lips smooth [title] Drink 8 glasses (1.9 liters) of water daily. [step] Staying hydrated is a great way to keep your lips moist, smooth, and wrinkle-free. Drinking water also has many other benefits, including clear skin.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6143,"error":null,"output":"B"}
{"question_id":"hellaswag:00001792","item_index":214,"attempt":0,"prompt_hash":"f3daaaeed132120f4e9853638d3aa18e06ccac74d0bd51816b4cdfdf3a08a438","question":"A group of young people play soccer in a large sand pit bordering a large green field. a group of young men and one woman\n\nA. does clips of the people playing soccer inside the sand mound as little girls watch on the sidelines.\nB. play soccer in a sand pit using their feet to kick and sometimes their heads.\nC. take turns swinging the ball over the net in three short volleys.\nD. talk to javelin players as they lay on the ground.","prompt":"A group of young people play soccer in a large sand pit bordering a large green field. a group of young men and one woman","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6144,"error":null,"output":"B"}
{"question_id":"hellaswag:00009996","item_index":215,"attempt":0,"prompt_hash":"3e03a89874c108b0e2283cea71ac5935c5f206d3d39d063881246df96bb4d069","question":"[header] How to clean granite countertops [title] Prepare countertops for proper cleaning. [step] Before you begin to actually clean the countertop, remove excess debris and move appliances or items so you can reach all areas. [substeps] Wipe up any spills or sticky residue on counter.\n\nA. The best way to avoid staining your countertop is to remove liquids or sauces immediately from your countertop. If the liquid has set, use a hot wet rag to loosen and remove the caked on debris.\nB. Wipe up any favorite color color or item. Also wipe up any spills in the cookware.\nC. Wash work surfaces with soap and water and scrub surfaces which have accumulated dirt. Prepare a new countertop surface when it becomes chipped, broken or clogged.\nD. Wipe up any spills with a damp kitchen sponge or rag. Use a garden hose to move cleaning supplies away from the countertop.","prompt":"[header] How to clean granite countertops [title] Prepare countertops for proper cleaning. [step] Before you begin to actually clean the countertop, remove excess debris and move appliances or items so you can reach all areas. [substeps] Wipe up any spills or sticky residue on counter.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6145,"error":null,"output":"A"}
{"question_id":"hellaswag:00000195","item_index":216,"attempt":0,"prompt_hash":"5cd4a1e7b97ccdf37fc459f64b2273a49949cf6e0c1ae16d40c478a545dfd4ff","question":"We see the title introducing us to jesse. Jesse the dog runs and catches frisbee. We see jesse close up in the camera. jesse\n\nA. gives instructions on how to clean a window in a home.\nB. is running around catching the frisbee the lady throws.\nC. throws the frisbee and a dog catches it.\nD. gives us the dog and we see the ending title screen.","prompt":"We see the title introducing us to jesse. Jesse the dog runs and catches frisbee. We see jesse close up in the camera. jesse","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6144,"error":null,"output":"C"}
{"question_id":"hellaswag:00006740","item_index":217,"attempt":0,"prompt_hash":"70d695b9589456a6a6966c8b7358b8dbc3a1cc4fd26a355756ae8318435e79f3","question":"[header] How to use a cleansing balm [title] Determine your skin type. [step] Cleansing balms can work well for any skin type, but are especially helpful for people with sensitive or dry skin because they are less drying. When you choose your cleansing balm, you want to make sure that it's formulated for the type of skin you have.\n\nA. It's always best to opt for a cleansing balm made from something you won't be using a lot of time or money on, since the drying process tends to be a lot more difficult for sensitive skin types. Select suitable cleansing balms for your needs.\nB. [substeps] If you've always had sensitive skin, rinse your face for five to 10 minutes in warm water. If you have different skin types, read reviews to figure out which one works for you.\nC. [substeps] If you have oily skin, using a cleansing balm with a soft cloth may help you prevent your skin from over-producing oils. If you have sensitive skin, go with a balm that doesn't have many ingredients.\nD. [substeps] You can find cleansing balm in many forms. Liquid and gel balms are good for oily people's skin; non-liquid balms are especially effective for those with oily skin.","prompt":"[header] How to use a cleansing balm [title] Determine your skin type. [step] Cleansing balms can work well for any skin type, but are especially helpful for people with sensitive or dry skin because they are less drying. When you choose your cleansing balm, you want to make sure that it's formulated for the type of skin you have.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6144,"error":null,"output":"C"}
{"question_id":"hellaswag:00005033","item_index":218,"attempt":0,"prompt_hash":"b4f12fac045f51693463e3ff5622ebf17cf8d209b3c3d327b6bfe6592430254b","question":"[header] How to calculate an amount to be financed [title] Determine the selling price. [step] For a vehicle, boat, or another type of commercial loan purchase this will be the amount you agree to pay for your new acquisition. It does not include other aspects of the deal such as the trade-in allowance, fees, taxes, and other closing costs.\n\nA. [title] Subtract any net trade-in allowance. [step] For auto or boat purchases, among others, a dealer may offer a trade-in allowance or credit for giving them your old car or boat when you buy a new one.\nB. Any negotiation will be slightly different, so be sure to talk with your broker about what you want and how much to get. [title] Do not make any forward decisions until you have narrowed your terms.\nC. Keep in mind that a profit made on a one-time purchase is dependent on the value of the product. The resulting value as a co-op loan will have a decline that is more noticeable if you purchase the vehicle outright, and be sought after.\nD. [title] Determine any fees or time limits you have for supplemental financing. [step] The rules regarding the amount purchased vary from state to state, and you should determine if you wish the required amount to be included in your purchase.","prompt":"[header] How to calculate an amount to be financed [title] Determine the selling price. [step] For a vehicle, boat, or another type of commercial loan purchase this will be the amount you agree to pay for your new acquisition. It does not include other aspects of the deal such as the trade-in allowance, fees, taxes, and other closing costs.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6144,"error":null,"output":"A"}
{"question_id":"hellaswag:00001211","item_index":219,"attempt":0,"prompt_hash":"6b0760ff8e4e9759c85c13edc6423cc65107a686a38d4e5ef6db057388a3f798","question":"The boy begins playing the drums. The boy continues playing the drums as smoke effects begin to appear. the boy\n\nA. then drops the drumsticks, poses, then walks off of the stage.\nB. stops playing and finds himself on a long floor.\nC. tries hard to keep the smoke inside but ends up coughing.\nD. continues to play the drums with the rest of his hands as smoke begins to enter on his face.","prompt":"The boy begins playing the drums. The boy continues playing the drums as smoke effects begin to appear. the boy","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6545,"error":null,"output":"D"}
{"question_id":"hellaswag:00006522","item_index":220,"attempt":0,"prompt_hash":"ec81d5c322eba41954310c309be78eb1558d05d2fad9b75bc39547e53e1b1ad9","question":"[header] How to grow early strawberries [title] Consider the average summer temperatures and level of humidity in your area. [step] Environmental factors like the average summer temperatures and the level of humidity will affect how well certain cultivars grow in your climate. Some cultivars thrive in hot, humid climates while others do better in dry, cooler climates.\n\nA. [title] Choose a location with good air circulation. [step] Ideally, a healthy way to grow strawberries is to choose a spot that receives direct sunlight.\nB. They do not flourish in dry, cloudy, or temperate climates, so you should do your best to plant the plants in low humidity spaces as well as in unhumid or cool areas. [substeps] Choose the best spot for the growing of the strawberries you wish to grow in.\nC. To determine the ideal climate for seedlings, look online or by walking by a local nursery or garden center. [substeps] Try to plant your strawberries in partial shade.\nD. [title] Purchase strawberry plants from your local nursery or garden center. [step] They should have a good selection of cultivars that will do well in your area.","prompt":"[header] How to grow early strawberries [title] Consider the average summer temperatures and level of humidity in your area. [step] Environmental factors like the average summer temperatures and the level of humidity will affect how well certain cultivars grow in your climate. Some cultivars thrive in hot, humid climates while others do better in dry, cooler climates.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6545,"error":null,"output":"D"}
{"question_id":"hellaswag:00004361","item_index":221,"attempt":0,"prompt_hash":"b71fe79e2ded7017fc7cab8508c2f796c12cf6970fc339f1b18558b95a7bb1c3","question":"[header] How to treat congestion with herbs [title] Use eucalyptus. [step] Eucalyptus is typically available as an oil instead of an herb. However, it is also available in lozenges, cough medicines, rubs, and bath salves.\n\nA. You can purchase the herb inexpensively from your local pharmacy or online. [substeps] Be sure to stock up on the herb you already use and one of its medicinal properties.\nB. [substeps] You can purchase it over-the-counter or you can make something with it. You can massage it into your sinuses with your fingertips or a cotton swab.\nC. [substeps] Eucalyptus makes digestion a natural part of normal life. It is commonly used to treat congestive heart failure.\nD. It has a strong scent and decongestant properties that can help with mucus build up. Eucalyptus has been proven safe for pregnant and nursing women as well as children and it has the gras status in the us..","prompt":"[header] How to treat congestion with herbs [title] Use eucalyptus. [step] Eucalyptus is typically available as an oil instead of an herb. However, it is also available in lozenges, cough medicines, rubs, and bath salves.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6544,"error":null,"output":"B"}
{"question_id":"hellaswag:00000078","item_index":222,"attempt":0,"prompt_hash":"a14d4270989008699a00ad69685dd6063f6a59e8a3eedcab163b56abc841a108","question":"We see a chair with a pillow on it. a man holding a cat\n\nA. does curling.\nB. starts hitting objects on an item.\nC. sits down on the chair.\nD. is wrapping a box.","prompt":"We see a chair with a pillow on it. a man holding a cat","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6544,"error":null,"output":"C"}
{"question_id":"hellaswag:00008227","item_index":223,"attempt":0,"prompt_hash":"cce205075ab4fda0537543742a6af94fa7c4b1f8530da9a781f4e8b20afd3653","question":"[header] How to paint a bubble design on your nails [title] Remove any existing polish off your nails. [title] Trim and shape your nails if needed. [title] Paint a clear base coat on your nails.\n\nA. [title] Continue painting your nails using the base coat. [title] Let your nails dry for at least 6 hours.\nB. [title] Slowly work your way from your toes up your nail and outwards. [step] Focus on the areas you typically want to cut or scuff.\nC. [step] This can help your nail polish last longer and reduce the chance of \" staining \" your nails. [title] Paint your nails the color you want the background to be.\nD. [title] Use nail paints to color red and purple nails. [title] Paint a medium to dark red on each nail.","prompt":"[header] How to paint a bubble design on your nails [title] Remove any existing polish off your nails. [title] Trim and shape your nails if needed. [title] Paint a clear base coat on your nails.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6544,"error":null,"output":"A"}
{"question_id":"hellaswag:00004817","item_index":224,"attempt":0,"prompt_hash":"3d38cbf2e5cfadf4561560a8be35bde6ee196675895528f7b5aacbfbfe72b0b2","question":"[header] How to be convincing [title] Make gestures and expressions. [step] Moving your hands and making facial expressions while you speak can make you seem confident and convincing. For instance, when you say something positive, smile a bit and open your eyes a little wider.\n\nA. If you sag when you say something negative, simply nod and smile to show that you're not unhappy with what you say. Show others that you're feeling whatever is \" bothering \" them by holding your eyes open and a little wider than you normally would.\nB. When you raise your eyebrows, frown and drop your shoulders once you're done, this shows you're thinking out loud, and can convince people. [substeps] Try practicing this by looking at a blank sheet of paper and trying to make eye contact with it while you say something.\nC. Of course, you can be transparent with people, especially yourself. However, think about the ways others see you.\nD. If you don't move at all while you are speaking, people might become suspicious. On the other hand, it is important to not to overdo it.","prompt":"[header] How to be convincing [title] Make gestures and expressions. [step] Moving your hands and making facial expressions while you speak can make you seem confident and convincing. For instance, when you say something positive, smile a bit and open your eyes a little wider.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6434,"error":null,"output":"D"}
{"question_id":"hellaswag:00008302","item_index":225,"attempt":0,"prompt_hash":"12a9a874ab2ae96f369b4b5c79019a0af06f7f1f519d7adb5388feae4dd50fbb","question":"[header] How to butter a brick [title] Grab the trowel. [step] Grab the trowel handle with your dominant hand. Your fingers should curl around the width of the handle, but your thumb should be extended along the length.\n\nA. [substeps] Holding the trowel in this manner will give you greater control over it. Your wrapped fingers should keep the trowel sturdy and secure in your hands, and your extended thumb should make it easier to control the direction you move the trowel.\nB. As your non-dominant hand covers the square and the two edges of the handle, use your other fingers to work the brick towards the center of the oven. [substeps] You'll see that the bottom section of the brick bends upward so that it's about halfway done.\nC. [title] Use blunt or stiff strokes to work the butter out of the brick. [step] Keep applying slow increments with the tool until the butter has completely been removed from the granite.\nD. How many inches is the handle wider than the tip of your palm? [substeps] The shank of your trowel is between the two front edges of the brick. Measure how thick you'd like your finish to be, then choose a medium length.","prompt":"[header] How to butter a brick [title] Grab the trowel. [step] Grab the trowel handle with your dominant hand. Your fingers should curl around the width of the handle, but your thumb should be extended along the length.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6435,"error":null,"output":"A"}
{"question_id":"hellaswag:00006944","item_index":226,"attempt":0,"prompt_hash":"912891d2b37196282ade903e085eb030adbaa723f75f953a277cc59dab1ef7a3","question":"[header] How to live in a haunted house [title] Take a well-built guard dog like german shepherd with you into the house. [step] The reason? Dogs easily detect the presence of the spirits. It will be daring to you also with the dog.\n\nA. Be aware that they are most likely to see ghosts through small dogs, but the sight of a dog walking around is also frightening. If you don't want to expose yourself to ghosts, just go with either a dog.\nB. [title] Remind yourself that before you move anything into the house, walk through the entire place. [step] Doing this will give the ghost (s) a chance to scope you out, and accept your presence in' their' home.\nC. Make sure the dog is not doing something dangerous, like scared to enter a haunted home, before undertaking more difficult things. [substeps] When other muggers remove a pet from the house, they will be difficult to kill.\nD. Be careful with this dog's stubbornness, of course. If the dog already has a grudge against a member of your family, it will quickly realize this is a dangerous situation.","prompt":"[header] How to live in a haunted house [title] Take a well-built guard dog like german shepherd with you into the house. [step] The reason? Dogs easily detect the presence of the spirits. It will be daring to you also with the dog.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6434,"error":null,"output":"B"}
{"question_id":"hellaswag:00004626","item_index":227,"attempt":0,"prompt_hash":"fe6c6b13b0c5d8d666f83b4f9f1de756aca9dc1ec93a2614caaf6ba44171d097","question":"[header] How to get motivated to write fan fiction [title] Read other fan fiction. [step] Whatever website you use for reading and posting fan fiction stories, most likely, it's good to read other people's work before submitting your own. It's a good source of inspiration and gets ideas flowing.\n\nA. [substeps] Go online and look over the characters and details of your favorite series or celebrities. It can help to begin a collection of fan fiction you can't find online, as there's a vast number of sites on the internet dedicated to this \" learn now \" approach to reading and writing fan fiction.\nB. [substeps] Read what other fans are reading or talking about. Watch fan fiction on social media, review youtube videos, just to name a few.\nC. If you're planning on writing a harry potter fan fic, it's probably a good idea to read over a short story or series of one shots of harry potter before you begin your story. [title] Read other material like part of a novel or a magazine article.\nD. [substeps] Consider talking to writers you know and looking them up online. They're more likely to tip you off to ideas you've never thought of, and they'll get to know you better.","prompt":"[header] How to get motivated to write fan fiction [title] Read other fan fiction. [step] Whatever website you use for reading and posting fan fiction stories, most likely, it's good to read other people's work before submitting your own. It's a good source of inspiration and gets ideas flowing.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6435,"error":null,"output":"D"}
{"question_id":"hellaswag:00002514","item_index":228,"attempt":0,"prompt_hash":"0fd2ae8be93cfc3140033fef8fbaabb9e448ff306ae408404dc6eae67015c925","question":"Several kids are playing beach volleyball on an enclosed, sand volleyball court. one of the boys\n\nA. hits it so well that it falls on the ground.\nB. walks over to the game to give a thumbs up then returns to the game.\nC. falls down after trying to go for the ball.\nD. from the curls stands in front of the camera.","prompt":"Several kids are playing beach volleyball on an enclosed, sand volleyball court. one of the boys","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6437,"error":null,"output":"C"}
{"question_id":"hellaswag:00008829","item_index":229,"attempt":0,"prompt_hash":"59053f290d7743be77c3493f320e1055ad797bd64a8d833f69354a45ddcbbd85","question":"[header] How to dry your painted nails quickly [title] Pick the right clean bowl. [step] You want a deep enough, and large enough, bowl that you can submerge the entire nail into the bowl. Ideally, you want a bowl large enough to dip all of your nails at once.\n\nA. You also want your bowl to have a deep enough bowl so you can submerge each nail. [substeps] Check the amount of time that you should soak the nail.\nB. [substeps] Pick a bowl that can stand up to freezing or chilling. Metal bowls will hold in the cold and be cold to the touch.\nC. [title] Fill a plastic water dish about two-thirds full with tap water. [step] Make sure that the bowl is big enough so that you can place every toe and nail before pouring in the water.\nD. You will be dipping each nail in a shallow bowl (like a frozen basin that you don't have to fill). [substeps] You want to only dip one nail in at a time; otherwise, your nails may not dry evenly.","prompt":"[header] How to dry your painted nails quickly [title] Pick the right clean bowl. [step] You want a deep enough, and large enough, bowl that you can submerge the entire nail into the bowl. Ideally, you want a bowl large enough to dip all of your nails at once.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6434,"error":null,"output":"A"}
{"question_id":"hellaswag:00008828","item_index":230,"attempt":0,"prompt_hash":"675ca61638492f42e93e1f09a210a7cf45d2b27f1ba7933b4374575c04ad98d2","question":"[header] How to perform nikah [title] Agree to get married. [step] The cornerstone of a nikah is a mutual agreement between two people to get married. Prospective spouses should fully discuss getting married before making any arrangements.\n\nA. Once you are ready, agree to get married as soon as possible. [title] Book a hotel and motel if you need to (campus, school, etc.\nB. [substeps] If you both travel, you might choose to have married in the pushup. This will make it easier for one another to adjust to the changes.\nC. [substeps] Ensure the local ordinances also allow for candidates to have children. [title] Agree upon a time and date when you will get married.\nD. If you want to be traditional, you may also have a formal proposal of marriage and consult your parents to make sure they approve. [substeps] Some mosques and imams may not perform nikah without the presence of parents or guardians, so parental approval may be especially important if you want to perform nikah in a more conservative mosque.","prompt":"[header] How to perform nikah [title] Agree to get married. [step] The cornerstone of a nikah is a mutual agreement between two people to get married. Prospective spouses should fully discuss getting married before making any arrangements.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6434,"error":null,"output":"D"}
{"question_id":"hellaswag:00003945","item_index":231,"attempt":0,"prompt_hash":"e5758195c6d54c9ce2241f50cb2b0646a3b1c2f69046ba695d6a55ac8ddf756f","question":"[header] How to become pope [title] Become a catholic. [step] To become pope, you must be both male and catholic. If you were not born into the catholic faith, you will need to convert.\n\nA. And if you don't, you will need to marry the person or church you wish to share pope status with. [title] Attend an annual prewedding ceremony.\nB. The process is called the rite of christian initiation. [substeps] This is a process that will take time.\nC. You need to believe in a religion and eat the right foods. To gain the right in-laws, you will need to be very christian.\nD. If you are a man and you want to become pope, you will need to do so. A commitment is made to jesus christ in order to become pope, and it is a very important step toward being pope.","prompt":"[header] How to become pope [title] Become a catholic. [step] To become pope, you must be both male and catholic. If you were not born into the catholic faith, you will need to convert.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6437,"error":null,"output":"B"}
{"question_id":"hellaswag:00001019","item_index":232,"attempt":0,"prompt_hash":"ed8a2cb0848f25a6efb918e7d825ac662657d65a7553ebd85e35c5148d8b7c62","question":"The man shovels snow for a third time. The man shovels a really long path of snow at a different location. the man\n\nA. turns that way, picks up the snow trimmer, and continues vacuuming and scrolling more of the impacts down towards the camera.\nB. trims snow off of leaves for the fourth time.\nC. sharpens a knife on a rock.\nD. turns around and shovels a second path of snow, right next to the recent shoveled snow path.","prompt":"The man shovels snow for a third time. The man shovels a really long path of snow at a different location. the man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6436,"error":null,"output":"D"}
{"question_id":"hellaswag:00004032","item_index":233,"attempt":0,"prompt_hash":"982ba1389ad5015f5d6b664dae4dae5fd7788ff487e8236872700fb97af0eefa","question":"[header] How to get everyone to do one's chores [title] Discuss and divide the chores. [step] Every household has certain chores that need to be done. The first step to making sure they are all done on a regular basis is to identify them all.\n\nA. Issue a preference for the chores that should all be done by each child. [substeps] You might say, \" we all need one chore each day.\nB. Sit down with all the members of your family who are old enough to do chores and work it out. Letting everyone have a say in their chores and getting them all to agree on their chores will help you get everyone to do them.\nC. From here, you can build a hierarchy for each of the chores that has to be delegated throughout the family. Let the children act as the leader, which they feel comfortable with.\nD. Disagreements about the chores should be evenly divided among the household. [substeps] Don't simply assign every chore to one child, invite them each time they do something they are not supposed to.","prompt":"[header] How to get everyone to do one's chores [title] Discuss and divide the chores. [step] Every household has certain chores that need to be done. The first step to making sure they are all done on a regular basis is to identify them all.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6434,"error":null,"output":"B"}
{"question_id":"hellaswag:00001880","item_index":234,"attempt":0,"prompt_hash":"1f5ec3a2285d7708fd72bd95a8a31405f273c84935ca905b3ee13f37812f08b9","question":"A close up of a lawn is seen followed by a person walking in holding a tool. the person\n\nA. continues raising the blades in the air and throws the tool into the grass.\nB. walks up a flight of steps with the tool and polishes the grass.\nC. then begins cutting up the lawn with the tool.\nD. turns around and walks back to his lawn.","prompt":"A close up of a lawn is seen followed by a person walking in holding a tool. the person","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6435,"error":null,"output":"C"}
{"question_id":"hellaswag:00003922","item_index":235,"attempt":0,"prompt_hash":"c521eed4658e960ad4dd29cf1494af70e64ce4c316fcff74974e0403170c00f6","question":"[header] How to make an oreo strawberry pie [title] Gather your ingredients. [title] Wash and cut strawberries into slices. [step] It does not matter how big or small the strawberries are they will all fit on the pie perfectly.\n\nA. [title] Chop the strawberries into small pieces. [step] Use a knife to mash the strawberries together.\nB. [title] Second, place enough strawberries to evenly cover the whole pie crust. [step] You shouldn't be able to see any of the pie crust but if you do it's not the end of the world.\nC. [title] Squeeze gummy worm seeds onto red paste for the filling filling once the beetles are out. [title] Take the frozen strawberries back and pop them in a large garbage bag.\nD. [title] Heat milk or margarine until it boils. [title] Mix heavy cream, egg, cherry extract and butter.","prompt":"[header] How to make an oreo strawberry pie [title] Gather your ingredients. [title] Wash and cut strawberries into slices. [step] It does not matter how big or small the strawberries are they will all fit on the pie perfectly.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6436,"error":null,"output":"A"}
{"question_id":"hellaswag:00006845","item_index":236,"attempt":0,"prompt_hash":"e93345ba4042508f5287071ffddd0404ea6d611fa893b994b9342d6c7726b078","question":"[header] How to add variable products to woocommerce [title] Log in to your wordpress dashboard. [step] This assumes that you already have the woocommerce plugin installed. You then need to have a look at your left hand menu and click on products] add product [title] Fill in the title for your product as you would do in a normal blog post.\n\nA. [step] Also add a description in the text body. [title] Scroll down the page, then by product data you have to select variable product.\nB. [step] Next, type in \" productivity \" without the quotation marks. Click and then put your own stylus somewhere near the title.\nC. [title] Click on the name of the product that appears in your blog. [step] It should be as specific as possible because it needs to focus the market to that particular product (p).\nD. [step] Be sure to delete any quotation marks that make it difficult to put in your new title. [substeps] Name the product when you add it and how long that will be.","prompt":"[header] How to add variable products to woocommerce [title] Log in to your wordpress dashboard. [step] This assumes that you already have the woocommerce plugin installed. You then need to have a look at your left hand menu and click on products] add product [title] Fill in the title for your product as you would do in a normal blog post.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6434,"error":null,"output":"A"}
{"question_id":"hellaswag:00007495","item_index":237,"attempt":0,"prompt_hash":"8b9853478c91fd22bb4d6f6fb2f498f4fc1cd48578a311a5b207f1d1d73531b8","question":"[header] How to take measurements (for women ) [title] Stand with your back straight in front of a full-length mirror. [step] Having good posture is the key to taking accurate measurements. [title] Wrap a soft tape measure around your bust.\n\nA. [step] Start where your bust meets your tailbone. [title] Measure yourself from your hips to your heels.\nB. [step] Start close to the navel; you will need to wrap a length of tape around it twice. [title] Take a ruler and measuring tape and make your marks in a perpendicular direction.\nC. [step] Take it around your back, across your shoulder blades and under your arms. It should wrap around the fullest part of your bust.\nD. [title] Pay special attention to your chest, waist, and legs. [title] Take your measurement just above your breasts or about a third of the way up the triangle.","prompt":"[header] How to take measurements (for women ) [title] Stand with your back straight in front of a full-length mirror. [step] Having good posture is the key to taking accurate measurements. [title] Wrap a soft tape measure around your bust.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6435,"error":null,"output":"C"}
{"question_id":"hellaswag:00004098","item_index":238,"attempt":0,"prompt_hash":"8f13b0710968ef77364410e2e5096b532b66f6f7fbe064d896703fa3b9df203e","question":"[header] How to use an otoscope [title] Be gentle with the patient. [step] The ear is a very sensitive organ and can injure easily if improperly examined. Avoid pulling, pushing, or generally being rough with the patient you are examining.\n\nA. This can calm your patient and minimize the risk of injury from sudden movements. [substeps] Ask your patient if the pressure is acceptable to them.\nB. Sometimes the patient can already complain of pain. [substeps] If possible, limit the session to less than one hour and avoid poking or moving the device at all times.\nC. Always provide a comfortable bottom to keep the patient from lifting their head. [substeps] The patient should be lying down with her head for the most part, with the mouth open and as close to their ear as possible.\nD. [substeps] If the patient is in pain, the eye feels like it is not open enough to receive light or sound. This puts strain on the ear and can obstruct the fisr and the magnifier when being examined.","prompt":"[header] How to use an otoscope [title] Be gentle with the patient. [step] The ear is a very sensitive organ and can injure easily if improperly examined. Avoid pulling, pushing, or generally being rough with the patient you are examining.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6436,"error":null,"output":"A"}
{"question_id":"hellaswag:00006535","item_index":239,"attempt":0,"prompt_hash":"779d95eac589e4891eaf09eb473fb60d5d5cdcf637004c3d4027ec8bdeb93875","question":"[header] How to tell when your parent is sad [title] Monitor any changes to their normal behavior. [step] Sadness will often cause some subtle changes in the way your parent acts. For instance, they might normally be very talkative, but lately they have been quiet.\n\nA. Pay attention to any marked changes in their usual behavior or routines. [substeps] Consider whether they are ruminating on an event or conversation.\nB. Now is an ok time to take a look at what's going on. [substeps] Instead of feeling bad, listen for any shifts to your parent's demeanor that may indicate sadness.\nC. And if they suddenly suddenly start doing everything else, then maybe they don't like to do it anymore. Understanding the change may help you a lot in identifying the cause.\nD. Their chatter can become so abrupt, and they might say, \" oh my gosh, mom, everything would look so unfair. \" listening to this kind of \" sweet talk \" with your parents will alert you when a change is taking place.","prompt":"[header] How to tell when your parent is sad [title] Monitor any changes to their normal behavior. [step] Sadness will often cause some subtle changes in the way your parent acts. For instance, they might normally be very talkative, but lately they have been quiet.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6436,"error":null,"output":"A"}
{"question_id":"hellaswag:00000853","item_index":240,"attempt":0,"prompt_hash":"88d1a6d571f932edb67858ce4b93088304f0ba6e0d129c30dedd263c0862aa2d","question":"A woman in a red workout shirt is talking about the benefits of an elliptical cross trainer. another woman in a pink tank top\n\nA. is talking about her trainer and the benefits of the machine.\nB. is demonstrating the use of the elliptical.\nC. is standing in front of her, assists her with the help of her waist.\nD. is talking about the benefits of being self sufficient for exercise.","prompt":"A woman in a red workout shirt is talking about the benefits of an elliptical cross trainer. another woman in a pink tank top","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6393,"error":null,"output":"B"}
{"question_id":"hellaswag:00002959","item_index":241,"attempt":0,"prompt_hash":"c775880a4c1bfeafbe60f668438dd6d3f2396585e03be2f1bd9f849bc493c7f8","question":"A garden is shown from the outside of a house. The camera turns to take in various plants and flowers. we\n\nA. then get a good look at the grass.\nB. see the shirt hanging on a white wall.\nC. see a man kneeling and lifting a lawn mower on a blue mat.\nD. see a family cutting up the bushes with a block cutter.","prompt":"A garden is shown from the outside of a house. The camera turns to take in various plants and flowers. we","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6391,"error":null,"output":"A"}
{"question_id":"hellaswag:00007733","item_index":242,"attempt":0,"prompt_hash":"cf84dfbd2d939b4efbd01d7aec7ce4e1e7dc3b10405e64223b949cf4f9f332f1","question":"[header] How to buy skin so soft [title] Open your web browser and navigate to avon's website. [step] Avon offers an online catalog with ordering capability. [title] Click the \" shop online \" option to access the online catalog.\n\nA. [step] This option will take you to a new page. [title] Click the words \" skin so soft \" in the black bar across the top of the page.\nB. [step] An email from avon will automatically be added to your profile page and you will be redirected to the shops page. [title] Find the product you want to purchase.\nC. [step] Items included in this catalog range from lightweight products to natural fibers. [title] Fill out the information contained in the catalog.\nD. [step] Avon orders frozen meals in the united states as well as meals at multiple scheduled locations from all over the world. Visit avon's website on a regular basis to schedule meals with listed restaurants at least once a month, if not more frequently.","prompt":"[header] How to buy skin so soft [title] Open your web browser and navigate to avon's website. [step] Avon offers an online catalog with ordering capability. [title] Click the \" shop online \" option to access the online catalog.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6388,"error":null,"output":"A"}
{"question_id":"hellaswag:00004045","item_index":243,"attempt":0,"prompt_hash":"13b55685e4da16cf177d7e4a09f7220407d86e44b725f35fdd8148db2c9bff0c","question":"[header] How to do tree braids [title] Find someone to braid your hair, or find someone that you can practice your tree braids on. [step] Unless you are very skilled at braiding, it would be very difficult to do this technique on yourself. [substeps] If you want someone to give you tree braids, offer to do a trade with them.\n\nA. Make sure the person is nicer than you. If they are not, they may not be the best person to braid your hair for you.\nB. [title] Decide what kind of twists to braid. [step] Once you have chosen your style, you can decide how long your braids should be.\nC. Suggest that you can give them braids if they do them to you as well. This way you both get practice and a new hairstyle.\nD. You can also try braiding other people's hair, as long as the seller doesn't accept credit. Ask someone to accompany you through the process, if possible.","prompt":"[header] How to do tree braids [title] Find someone to braid your hair, or find someone that you can practice your tree braids on. [step] Unless you are very skilled at braiding, it would be very difficult to do this technique on yourself. [substeps] If you want someone to give you tree braids, offer to do a trade with them.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6388,"error":null,"output":"A"}
{"question_id":"hellaswag:00000580","item_index":244,"attempt":0,"prompt_hash":"c9aeb4fe32adf6a744cb319ed2e05e3f3e625fcab8361d1826f158e46a764b9e","question":". a counter top with lemons and ingredients in small bowls\n\nA. is stacked up against a wall.\nB. are floating on a dark blue body of water.\nC. is shown, with several individuals standing by the counter.\nD. , a long piece of braided yarn inside, and a little black pail full of apples on a table sitting next to a long wooden bench in front of a white wall on metal, metal table, wooden table, wooden bench, beanbag chairs, wooden kitchen countertop and glass red shade top sponge and white bowl full of chopped apples and green strawberries.","prompt":". a counter top with lemons and ingredients in small bowls","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6389,"error":null,"output":"C"}
{"question_id":"hellaswag:00001875","item_index":245,"attempt":0,"prompt_hash":"b39cbe17e949e7be42dc313c595ebd6101ad37651315176c9a4ef33938412e9c","question":"Two men are seen playing a game of badminton with one another and leads into one of them speaking to the camera. the man\n\nA. continues to speak to the camera while showing off a set of perfect shot penises and using a pencil to draw in designs.\nB. continues to speak to the camera and leads into several clips of him speaking to the camera.\nC. then demonstrates how to properly hit the birdee while moving in slow motion and leading back into him speaking and playing with others.\nD. leads the other back and forth while the lead mans shakes his fists and ends by holding a white towel from behind him and speaking to the camera.","prompt":"Two men are seen playing a game of badminton with one another and leads into one of them speaking to the camera. the man","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6389,"error":null,"output":"B"}
{"question_id":"hellaswag:00001776","item_index":246,"attempt":0,"prompt_hash":"c6f0f55cd2406b34328e09ec254a713024b99670c11a26225b49de6942483398","question":"The man then grabs the knife and starts gliding it over the block very gently and smoothly before hitting it over a silver steel stick. when finish\n\nA. and the knife is seated back into the black case, the man takes it out again and writes on the back of the case along with the words \" pizza! vasquez cooking \" and from the words \" gemma's recipe \" he takes a look at the ingredients and then writes and separates them neatly.\nB. , the object is removed from the back of the refrigerator and shown to be a neatly cleaned bbq fry pan.\nC. , he grabs a piece of paper and shows how it cuts so effortlessly.\nD. its only the knife's edge.","prompt":"The man then grabs the knife and starts gliding it over the block very gently and smoothly before hitting it over a silver steel stick. when finish","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6389,"error":null,"output":"C"}
{"question_id":"hellaswag:00003459","item_index":247,"attempt":0,"prompt_hash":"0e5a2189590c699b554cb136910b1ae873ec8fe29dfe7224c8f0f39c184db5a4","question":"[header] How to get kids to behave in the supermarket [title] Be consistent with your behavior expectations. [step] Wherever you're going, it helps children to know how they're expected to behave, especially in public. Set some ground rules that you hold them to always, including at the supermarket.\n\nA. This lets your children know that you're aware of their behavior and which you can enforce. [substeps] Be clear about which foods are good, which are bad, and which are not.\nB. The food you serve the kids to pick which food is best for their age group. Make the rules so your kids can think clearly about what goes in and what comes out, so they aren't distracted from their schedules.\nC. Be understanding and reasonably flexible, but don't allow bad behavior to become an exception or habit. [title] Inform your child where you will be going.\nD. [title] Label what food you're serving. [step] Children need to know what's allowed.","prompt":"[header] How to get kids to behave in the supermarket [title] Be consistent with your behavior expectations. [step] Wherever you're going, it helps children to know how they're expected to behave, especially in public. Set some ground rules that you hold them to always, including at the supermarket.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6387,"error":null,"output":"C"}
{"question_id":"hellaswag:00000896","item_index":248,"attempt":0,"prompt_hash":"b74c5bf5906c23d7e3c70cbfe8917042919086a0b79ed14d6e19efb2e7c9db09","question":"There's a lawn trimmer lying on the ground near some over grown patches of grass. a person wearing gray pants and gloves\n\nA. using a lawn trimmer to trim the patches of grass.\nB. is cutting the grass with a yellow, reusable light green plastic tool.\nC. is nailing up a small patch of grass.\nD. cuts the grass with a hose and computer monitor over time.","prompt":"There's a lawn trimmer lying on the ground near some over grown patches of grass. a person wearing gray pants and gloves","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6389,"error":null,"output":"A"}
{"question_id":"hellaswag:00008813","item_index":249,"attempt":0,"prompt_hash":"e13c6774094c7a97625fa4dba27f1719827a4933337a24ddf64127d4309d9f66","question":"[header] How to decide if blonde hair is right for you [title] Consider your everyday lifestyle. [step] Blonde hair comes with a more sensitive upkeep. For example, with blonde hair, you can't jump right into a pool.\n\nA. These days, with blonde hair, you're unable to have casual conversations, workout, and get out of bed unless you exercise, run or do a lot of physical activity. [substeps] Blonde hair comes with a lot of hairstyles! Experiment with different hairstyles until you find one that suits you.\nB. Instead, you need to decide if your daily hair maintenance regimen will compromise your personal hygiene. [substeps] Blonde hair should be warmed up and air dried, rather than heat-dried as will women.\nC. Blonde hair absorbs chlorine and other chemicals that will give it a green appearance after swimming. Also, blonde hair that is washed in a shower where the water is high in copper or iron will eventually start to look orangy-red or green.\nD. Therefore, you may have to get your hair wet twice a day, hair shampooed twice a day, and body washed twice a day. Conversely, you may need to do these things two nights a week to model your look with your hair in a flattering condition.","prompt":"[header] How to decide if blonde hair is right for you [title] Consider your everyday lifestyle. [step] Blonde hair comes with a more sensitive upkeep. For example, with blonde hair, you can't jump right into a pool.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6388,"error":null,"output":"C"}
{"question_id":"hellaswag:00001758","item_index":250,"attempt":0,"prompt_hash":"1c22f380b1c3e1ff6b5eca0d39de01c28c454f5ccbf6fabf2c0f15c88d84f86a","question":"A man with a blue shirt, blue sweat pants and green socks is standing outside in a large field walking into a circle of cement. once the guy\n\nA. is inside, he extends a hand and begins hitting a wall.\nB. is inside, the man begins spinning in circles and putting the pipe in his mouth.\nC. is on the circle, he begins walking around the circle to find his perfect spot.\nD. has done some stretching the man starts using and picking up metal scraps.","prompt":"A man with a blue shirt, blue sweat pants and green socks is standing outside in a large field walking into a circle of cement. once the guy","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6387,"error":null,"output":"C"}
{"question_id":"hellaswag:00006184","item_index":251,"attempt":0,"prompt_hash":"c5f784220d59b83c8c204228209522e6b296a2682bf43b1638e3941eca87cc76","question":"[header] How to take off false eyelashes [title] Purchase a false lash glue remover. [step] Most false eyelash kits come with eyelash glue, but they don't often contain glue remover. However, you can find many brands of eyelash glue remover in the beauty department of your local department store.\n\nA. [substeps] Pay close attention to the type of glue, which differs from product to product. A commercially produced eyelash glue remover can produce a super thick paste that does not even bother people who wear false eyelashes.\nB. [substeps] If you can't find eyelash glue remover, try going to a drugstore that sells eyelash glue removers. [title] Put the frozen fixer on the lash-lifter and remove the lid.\nC. [substeps] When buying a false eyelash glue remover, place a stick of glue on the upper lash line. You can then wipe the stick of glue off.\nD. Choose a brand that meets your specific skin requirements; when possible, choose glue remover made by the same company that made the eyelash glue you used. [substeps] Always check to make sure you're not allergic to any of the ingredients.","prompt":"[header] How to take off false eyelashes [title] Purchase a false lash glue remover. [step] Most false eyelash kits come with eyelash glue, but they don't often contain glue remover. However, you can find many brands of eyelash glue remover in the beauty department of your local department store.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6037,"error":null,"output":"D"}
{"question_id":"hellaswag:00000809","item_index":252,"attempt":0,"prompt_hash":"ed272ade3354811b52b06cae0b1dbcbcf3f0dacb6505abf301c92ef7813406df","question":"Five people are jump roping on a blue mat. they\n\nA. perform flips and backwards handstands.\nB. sit at a table talking and laughing.\nC. do flips and tricks while jump roping.\nD. jump on top and harness up.","prompt":"Five people are jump roping on a blue mat. they","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6036,"error":null,"output":"C"}
{"question_id":"hellaswag:00006350","item_index":253,"attempt":0,"prompt_hash":"76850cd15009a7a2463e9e005c592c6a0e1577838013d0312b94f9c752fe19b2","question":"[header] How to stop labeling yourself as an underachiever [title] Look closely at your expectations. [step] If you are labeling yourself as an underachiever, this means that you are accomplishing less than you expect. So it is possible this lies not within your work, but instead in your own expectations.\n\nA. In fact, you may not even be fully aware of the high expectations you have for yourself. Take some time to truly reflect on what you think you should be accomplishing.\nB. Underestimating your own achievements is one of the most common reasons people share idealised attitudes toward a given academic subject. [title] Think about the reasons why you want to change the viewpoint.\nC. [substeps] In order to fully identify if you are making a false assumption, make a list of the things you expect to achieve and how many you expect to accomplish in a given experience. This list will guide your labeling abilities as you evaluate your own assumptions.\nD. [substeps] If you expect your work to go more smoothly if you are unreasonably stressed out, ask yourself why. Is your productivity slowing or worsening? Have you stopped driving? Have you crossed the street? Have you fallen victim to loneliness? Remind yourself that you are deserving of all that you are given.","prompt":"[header] How to stop labeling yourself as an underachiever [title] Look closely at your expectations. [step] If you are labeling yourself as an underachiever, this means that you are accomplishing less than you expect. So it is possible this lies not within your work, but instead in your own expectations.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6037,"error":null,"output":"A"}
{"question_id":"hellaswag:00000707","item_index":254,"attempt":0,"prompt_hash":"62ba14f341d22055fdc9bc3f731d9b23dd5f006d9ad81dd295fe9fea6ed86fee","question":"A man is seen spinning a blade with his foot on a machine and moving his hands up with down holding a knife. the camera\n\nA. then pans around to a woman and her daughter who also dance at the show.\nB. captures him from several angles while he sharpens the knife with complete concentration.\nC. pans around and points to a man standing inside the machine as the man continues to move on the machine.\nD. pans around and shows a woman moving around in a jump rope machine.","prompt":"A man is seen spinning a blade with his foot on a machine and moving his hands up with down holding a knife. the camera","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6036,"error":null,"output":"B"}
{"question_id":"hellaswag:00006757","item_index":255,"attempt":0,"prompt_hash":"a796bb6baaa8ce9a1e4c2b91eae9860d2284b81cf6e9d58b5996f9856b4ce9e3","question":"[header] How to always look attractive [title] Maintain a healthy diet. [step] Being as healthy as possible is an important part of being attractive. Make sure to drink a lot of water-at least 8 cups a day is advisable.\n\nA. Your body physically is composed of adimusculature and your metabolism needs to increase properly. [substeps] Add fresh fruits and vegetables to your diet in small increments.\nB. Avoid trans fats, sugary beverages and prepackaged foods, as well as junk foods. [substeps] Also stick to low-sugar fruits and vegetables.\nC. Keeping yourself hydrated is extremely important for being attractive. [substeps] Try eating fruits and vegetables on a regular basis, but keep in mind that fruits and vegetables often have little nutritional value.\nD. Eat fruits and vegetables on a daily basis. If you eat meat, try to stick to lean protein-like chicken, fish, and turkey.","prompt":"[header] How to always look attractive [title] Maintain a healthy diet. [step] Being as healthy as possible is an important part of being attractive. Make sure to drink a lot of water-at least 8 cups a day is advisable.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6037,"error":null,"output":"C"}
{"question_id":"hellaswag:00008122","item_index":256,"attempt":0,"prompt_hash":"7c70c174d01da0f889990631242f1105550aa4bd030b99881c5ca88b118c96b6","question":"[header] How to throw a surprise birthday party [title] Determine if your honoree will react well to a surprise. [step] Not all people react well to something unexpected. So before planning a surprise party consider how your guest of honor will react.\n\nA. Just like many college-aged people, it's important to gather some pros and cons to determine what a surprise birthday party is for your guest. [substeps] If your honoree has a hard time being surprised, have a surprise surprise party if the guest of honor reacts the best.\nB. The more they react, the more pleased they will be. [substeps] Plan a surprise party on a weekend in the summer in august, or on the weekends in september or march.\nC. Some may be more mentally and emotionally angry than others. Additionally, some may be curious or critical.\nD. If you are unsure, talk to their close friends or family members to feel out the party idea before proceeding. Also consider whether they enjoy parties or like being the center of attention.","prompt":"[header] How to throw a surprise birthday party [title] Determine if your honoree will react well to a surprise. [step] Not all people react well to something unexpected. So before planning a surprise party consider how your guest of honor will react.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6245,"error":null,"output":"D"}
{"question_id":"hellaswag:00001567","item_index":257,"attempt":0,"prompt_hash":"a78fec46cb7196fe33247fb5c8bd82e40c350a13f8cbb7d2d9a37e99f9c31257","question":"She is now going back and forth through another obstacle of sticks and runs towards the woods where she stops and points. she\n\nA. holds her hand out in front of her face letting the smoke thing become visible.\nB. then returns to calm water.\nC. runs all the way back down the field, giving high fives to random people on the field at her arrival.\nD. then goes back to where she started and is once again walking by herself.","prompt":"She is now going back and forth through another obstacle of sticks and runs towards the woods where she stops and points. she","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6249,"error":null,"output":"D"}
{"question_id":"hellaswag:00004762","item_index":258,"attempt":0,"prompt_hash":"9e41435875b8ef6ff13683abbb3c92492dd5d55d0ba8eb42d41b64843a607956","question":"[header] How to be honourable [title] Be the person you say you are. [step] It's easy to be a pleasant person, walking around with a ready smile and a \" hello \" for everyone you see. But being honourable isn't the same thing as being friendly.\n\nA. Being sir or madam isn't for everyone, but not every person just like you. Being classy in a respectful and accepted way is the last thing someone likes.\nB. While politeness is a necessary component of being honourable, it's not how you say' hello' or' goodbye', but what you say' goodbye' for. Here are some ways to be honourable : [substeps] Don't send mixed signals.\nC. When it comes to honor, it's more important to be authentic. Show the world who you really are, even if it come at the expense of your reputation for being \" nice.\nD. If your with a friend or relative, ask them if they would like a coffee after work or if they would like to chat with you. If you want to be honourable with a family member, know that they may think you don't deserve to be treated, respect them, and try to spread goodwill and cheer you up.","prompt":"[header] How to be honourable [title] Be the person you say you are. [step] It's easy to be a pleasant person, walking around with a ready smile and a \" hello \" for everyone you see. But being honourable isn't the same thing as being friendly.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6245,"error":null,"output":"C"}
{"question_id":"hellaswag:00002824","item_index":259,"attempt":0,"prompt_hash":"19b25840ac8ddd2609719190f6ce3441550c46106d6bbf59cda46b1d1bc5aeb0","question":"The same young lady is now standing in a different area holding a hand towel as she practices her swings and words on the screen give tips. the girl\n\nA. is still using her standing hand towel and the left hand has a different device while the right hand holds the stick and the words and text on the screen list.\nB. is back on the concrete circle swinging her disc and she then throws the disc.\nC. now is shown in a different environment with only her legs and her arms and she is able to do the movements in her swing without using a pole.\nD. 's standing in the exact same spot and she raises her right hand while smiling in her feet while jumping each time she swing.","prompt":"The same young lady is now standing in a different area holding a hand towel as she practices her swings and words on the screen give tips. the girl","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6248,"error":null,"output":"C"}
{"question_id":"hellaswag:00001372","item_index":260,"attempt":0,"prompt_hash":"7f1deb55e43ff1a4e84d90182af6171587ad3d6d9f5adeeb25563c2c541ffddf","question":"The women both put something from a cup in there mouths. The woman in red begins to dance. the woman in black\n\nA. begins to dance alone.\nB. finishes the dance and she's wearing something white that melts.\nC. makes a face to the camera.\nD. shows the other woman how to use the hookah.","prompt":"The women both put something from a cup in there mouths. The woman in red begins to dance. the woman in black","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6247,"error":null,"output":"C"}
{"question_id":"hellaswag:00007615","item_index":261,"attempt":0,"prompt_hash":"f2b7b87ddc0a63e6d60e0720542416f6870d5acb1fcb5b58ec2b6717ae8f4a34","question":"[header] How to clean brushed stainless steel [title] Start with water. [step] For basic spills, just plain water will do the trick. Use a soft rag, and apply warm water to the surface.\n\nA. You can use this method to wipe up spills, as well as small stains. Wipe with clear water once the stain is up.\nB. Rub it in very around the buffed areas. It should penetrate through the dirt on the buffed area and buff away the grime.\nC. This helps to give it a good soak so it isn't too wet. [substeps] Repeat this process with dish soap, dry alcohol or vinegar for extra cleaning power.\nD. Then, do some quick clean-up with a sponge and some conditioner. Make sure to get rid of any slime and oil that's left behind with the rag and conditioner.","prompt":"[header] How to clean brushed stainless steel [title] Start with water. [step] For basic spills, just plain water will do the trick. Use a soft rag, and apply warm water to the surface.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6247,"error":null,"output":"A"}
{"question_id":"hellaswag:00009761","item_index":262,"attempt":0,"prompt_hash":"36c5164432ac9b946cae985d119d9e609d221d102191f3c5dda743377bed5c4a","question":"[header] How to trust a wife [title] Trust yourself. [step] Trust that you are good husband. Trust that you are in the right place, at the right time, and experience the right experience.\n\nA. In love and love is a special feeling. [title] Know that you are in an important relationship.\nB. Each experience teaches you to become a better person. If your wife cheated, lied, or betrayed you, trust yourself that you are a strong man.\nC. Don't do any guilt over your actions or behavior, but remember that you are a man who is functioning well. Wear clothes that are nice-looking and that you feel comfortable in.\nD. Trust that you will not regret the experience. [title] Take care of her, keep up your promises, and do the right thing.","prompt":"[header] How to trust a wife [title] Trust yourself. [step] Trust that you are good husband. Trust that you are in the right place, at the right time, and experience the right experience.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6247,"error":null,"output":"B"}
{"question_id":"hellaswag:00004685","item_index":263,"attempt":0,"prompt_hash":"156c07a7599d911a88e8112ca21ac9138bec3248b9e5fb7577e776bf3f9b277d","question":"[header] How to keep your old phone number [title] Contact your service provider. [step] Telephone companies allow transfer of telephone lines from one place to another. What's going to happen is that they'll just transfer your existing line to the area where you're moving to instead of creating a new telephone line.\n\nA. You'll have to talk with your provider about where to find your line, and do this by logging into the websites of any other service providers. [substeps] Click on the \" new line \" link.\nB. This means you're not transferring an existing line altogether. For instance, if your home is in the far west, you might turn your old line into a toll-free number.\nC. Making sure you still have your old number makes things easier on yourself. [substeps] Call * 41-785-9141 to inquire at the station where you last spoke.\nD. On the other hand, not all landlines can be transferred for these reasons : [substeps] The area you want your landline to be transferred is not covered by your telephone company. The telephone company may not have any existing service for the area where you want your phone to be transferred to.","prompt":"[header] How to keep your old phone number [title] Contact your service provider. [step] Telephone companies allow transfer of telephone lines from one place to another. What's going to happen is that they'll just transfer your existing line to the area where you're moving to instead of creating a new telephone line.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6244,"error":null,"output":"D"}
{"question_id":"hellaswag:00009734","item_index":264,"attempt":0,"prompt_hash":"1d4ccbb9f0816cde780a1b66ae5c33ae5e4cc672c5e10802a1a060ae0e0af47b","question":"[header] How to increase chest hair [title] Level out your testosterone. [step] Talk with your doctor about whether you have normal testosterone levels. Testosterone has a peculiar relationship to hair loss.\n\nA. Not enough testosterone and your chest hair won't grow. Too much and it gets converted into dht, which shrinks hair follicles.\nB. Polycystic ovary syndrome (pcos) is a genetic disorder that affects men and women. Pcos levels appear as a genetic problem.\nC. Women with either bad hormones or difficulty maintaining a healthy body will have low levels of testosterone and can thin hair more quickly. Women with an untreated hair loss history or conditions found in hair loss will have low levels.\nD. If you do not maintain your testosterone levels level even a little bit, it may be cause for concern to you. Your doctor can help you adjust to the changes.","prompt":"[header] How to increase chest hair [title] Level out your testosterone. [step] Talk with your doctor about whether you have normal testosterone levels. Testosterone has a peculiar relationship to hair loss.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6246,"error":null,"output":"A"}
{"question_id":"hellaswag:00000088","item_index":265,"attempt":0,"prompt_hash":"1b8108c23816714b22bd1161be2e2fbb54ca52e7d8a798c224448daa50a8b81c","question":"A man is seen throwing a dart onto a board and leads into a woman speaking. the man\n\nA. tosses darts into the water while people measure the distance and run after them.\nB. gives her several tips and she continues creating darts.\nC. throws several more darts while taking the time to speak to the woman.\nD. holds behind him and the woman continues speaking.","prompt":"A man is seen throwing a dart onto a board and leads into a woman speaking. the man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6245,"error":null,"output":"C"}
{"question_id":"hellaswag:00007961","item_index":266,"attempt":0,"prompt_hash":"587992243c5f35fe49b367bbc4919c00e26dba7cfb9f6c53c059a2a044a8b982","question":"[header] How to get a quit claim deed [title] Seek help from an attorney or title company. [step] Your state or county may have a \" fill in the blank \" quitclaim deed you can use. Stop into your county's land records office to check.\n\nA. If you are unsure how to obtain a quitclaim deed, you may file your lawsuit in a form specific to your claim. It generally is advantageous to call 1-800-375-1213 and ask.\nB. [substeps] If you are a sole proprietorship, you can request a copy of title's quitclaim deed. Most state bar associations offer log books for independent and neutral parties to legal documents.\nC. [substeps] For example, in california, you can get quitclaim deeds from the california california department of land management and registration. [title] Fill out a federal quitclaim deed.\nD. This office is probably called the recorder of deeds office or something similar. A form may also be available on the county court website.","prompt":"[header] How to get a quit claim deed [title] Seek help from an attorney or title company. [step] Your state or county may have a \" fill in the blank \" quitclaim deed you can use. Stop into your county's land records office to check.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6243,"error":null,"output":"D"}
{"question_id":"hellaswag:00003393","item_index":267,"attempt":0,"prompt_hash":"268fdafbcf0351383902fab3d9cc38f21100ed19e3beb02a1ff0ffdf88612824","question":"[header] How to clean tile with vinegar [title] Sweep or vacuum the floor. [step] Before cleaning a tile floor with vinegar, you will want to remove all debris from the surface of the floor. Use a broom, dust mop, or vacuum cleaner to remove all loose dirt and debris from the tile floor.\n\nA. Once you are done, wipe up any excess dirt by brushing the floor with a soft bristled brush or nylon bristled duster. [substeps] Vinegar has long been used as a household product for its abrasive properties.\nB. Ensure that you have a maximum of 60% vinegar stain from the cleaning cleaner and vacuum up any loose debris. Allow the stained area 24 hours to air dry and sweep and mop the floor.\nC. [substeps] Sweep the floor every week to ensure any lingering dirt remains on the floor. If you only want to clean the floor, sweeping the floor daily instead of once per week will take up significant amount of time.\nD. [substeps] Vinegar is great for ceramic tile floors. Don't use vinegar on marble or granite.","prompt":"[header] How to clean tile with vinegar [title] Sweep or vacuum the floor. [step] Before cleaning a tile floor with vinegar, you will want to remove all debris from the surface of the floor. Use a broom, dust mop, or vacuum cleaner to remove all loose dirt and debris from the tile floor.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6243,"error":null,"output":"D"}
{"question_id":"hellaswag:00006138","item_index":268,"attempt":0,"prompt_hash":"dd0316e1b6d074385f2292d85deea39e98bb25ac67709a4811b538e68a7da997","question":"[header] How to collect cards at your wedding [title] Use a secure card box. [step] If you choose to collect cards using a card box, make sure it is secure. Choose a card box that has a lock.\n\nA. Then, use your card box's key combination to secure it. Put the key combination in the \" + \" on your card box or off to the side.\nB. The card box should have a single slit in the top. It should remain locked throughout your wedding festivities.\nC. This will make it easier to access your cards at the wedding. [substeps] There are a number of card boxes that you can purchase at your wedding shop, but any one that you choose needs to be secure.\nD. This provides security for your guests who want to know the card contains secrets. [substeps] If you add valuables into a box, make sure there are locks at the bottom of it.","prompt":"[header] How to collect cards at your wedding [title] Use a secure card box. [step] If you choose to collect cards using a card box, make sure it is secure. Choose a card box that has a lock.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6245,"error":null,"output":"B"}
{"question_id":"hellaswag:00002697","item_index":269,"attempt":0,"prompt_hash":"feb351eb9f4caa84b0705f58194fb9d44c8374b0638d5d075c3ceb54d68ace66","question":"Two people are seen riding down a road with a dog. one person\n\nA. is riding a bike and the other a skateboard while the camera follows close behind.\nB. is seen walking down the road alone.\nC. spins around the dog while the other walks along behind.\nD. chases the other down and shows clips of a dog running along the road.","prompt":"Two people are seen riding down a road with a dog. one person","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6244,"error":null,"output":"A"}
{"question_id":"hellaswag:00004832","item_index":270,"attempt":0,"prompt_hash":"90555862f9208a0cfcd4551c210e735cbc9b263155b9a9d55865db26da6a6dca","question":"[header] How to act around the girl you like [title] Build your confidence. [step] Confidence is one of the most attractive qualities a person can possess. You don't have to be super sexy or built like chris hemsworth, either.\n\nA. Your poise, ability to adapt to situations and a certain amount of self-control is important for acting confident. Also, you don't have to sport an unkempt appearance to draw the girl to you.\nB. Come on, this girl really like you, at least if she isn't conceited. But embrace that little sassy girl who stands up for her friends, who stands up for those in the way she sees fit.\nC. Continue to show confidence and a bit of other things that'll make you feel attractive. [substeps] Don't try to be one of the girls everyone else thinks you are; put yourself out there, and the girl will notice.\nD. All confidence requires is a firm belief in your own self-worth. If you believe it, that girl you like? She's going to believe it, too.","prompt":"[header] How to act around the girl you like [title] Build your confidence. [step] Confidence is one of the most attractive qualities a person can possess. You don't have to be super sexy or built like chris hemsworth, either.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6245,"error":null,"output":"D"}
{"question_id":"hellaswag:00005358","item_index":271,"attempt":0,"prompt_hash":"b3bde4779117129076b2de54e705ca0c1798e1cf3e9d2e5ed2423d4f302329b1","question":"[header] How to review an individual transition plan (itp), autism spectrum disorder [title] Ensure that the itp begins early in the student's life for it to be effective. [substeps] They are complex and require significant time to teach, absorb and become effective for life after school. Many schools do not start them until the student is fifteen or sixteen years old.\n\nA. This will require many subtests with lots of parts to write, problems and effective approaches to solving/analysing issues early. They may be approved by either your school, religious institutions such as abb must agree with the guidelines, not yours before that.\nB. It is therefore very important that, in order to be effective for everyone, itp must start at the ten minute mark. Even if the student seems competent now in year time, this does not necessarily mean that they are not capable in years later.\nC. Once the student has reached that age, their individuality changes greatly, and characterization is accelerated. [title] Have a mentor monitoring the patient's progress.\nD. For students with asds, this often does not provide enough time. The itp process can begin at the age of thirteen, or earlier if need indicates that the need is present, and can follow through until the age of twenty one.","prompt":"[header] How to review an individual transition plan (itp), autism spectrum disorder [title] Ensure that the itp begins early in the student's life for it to be effective. [substeps] They are complex and require significant time to teach, absorb and become effective for life after school. Many schools do not start them until the student is fifteen or sixteen years old.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6244,"error":null,"output":"D"}
{"question_id":"hellaswag:00003785","item_index":272,"attempt":0,"prompt_hash":"f7787805929afa6db95422aee8ce27173a0ab85300fdd403c66f6dcb359c3e4c","question":"[header] How to install a window well [title] Take your prefabricated window well and place it against your ground-level window to get an idea of the area you need to dig out. [step] Mark a perimeter of about 6 inches (or 15 cm) around the well with your shovel as a guide for digging. [title] Dig your hole.\n\nA. [step] With your clay-head shovel, begin digging into the ground about 3 feet (0.91 m) deep, at a 45 degree angle. Continue digging in a row to ensure your hole is even with the outside of the window well.\nB. [step] If you are digging down using a tractor, with a hoe, or with a box hoe, you may need to start with the underside of the box hoe. Be sure that you do not dig into anything on top of the hole.\nC. [step] Use long, metal nail which you can get at auto parts stores. Dig at a wide depth, often about 2 feet (0.6 m), and never bury more than 3 inches (7.6 cm) deep.\nD. [substeps] The top of the well should sit about 6 inches (or 15 cm) above the ground. You should also allow about 4 inches (or 10 cm) for gravel beneath the well.","prompt":"[header] How to install a window well [title] Take your prefabricated window well and place it against your ground-level window to get an idea of the area you need to dig out. [step] Mark a perimeter of about 6 inches (or 15 cm) around the well with your shovel as a guide for digging. [title] Dig your hole.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6682,"error":null,"output":"D"}
{"question_id":"hellaswag:00004187","item_index":273,"attempt":0,"prompt_hash":"57ade1d94345865c7bfe9ca5fcfbea822c09fb8e6ce32beb3527c25d088e407e","question":"[header] How to wear high waisted shorts [title] Go for shorts that hit below your belly button if you are petite. [step] Try high-waisted shorts that hit a few inches below your belly button so they do not swallow up your torso. Shorts with a top button that hits just below your belly button may make your torso appear shorter, rather than elongate your frame.\n\nA. [substeps] The top button should also be made of high-quality cotton-linen or other synthetic materials. Most styles of high waisted shorts are right above the waist.\nB. [title] Wear shorts with shorts that jut below your knees. [step] Wearing shorts with a top button on your bottoms can keep your torso looking more slender as you wear them out.\nC. [substeps] You may need to try one several pairs of high-waisted shorts with different inseam lengths to find the right one for your petite frame. [title] Try shorts with an a-line cut if you are curvy.\nD. [title] Wear high-waisted shorts when you're skinny. [step] High-waisted shorts let you comfortably walk down the beach.","prompt":"[header] How to wear high waisted shorts [title] Go for shorts that hit below your belly button if you are petite. [step] Try high-waisted shorts that hit a few inches below your belly button so they do not swallow up your torso. Shorts with a top button that hits just below your belly button may make your torso appear shorter, rather than elongate your frame.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6682,"error":null,"output":"C"}
{"question_id":"hellaswag:00008744","item_index":274,"attempt":0,"prompt_hash":"d8ea774e0bb4b2ead1e5e5aac81f8930e5be0ad00b0cf123a7d8e09c4ed32968","question":"[header] How to melt chocolate chips [title] Fill a saucepan with 2 inches (5.08 centimeters) of water. [step] You can use an actual double boiler, or improvise your own with a saucepan and a heat-safe bowl. Do not place the second saucepan or bowl on top just yet, however.\n\nA. [title] Melt one cup (240 milliliters) of milk in a saucepan over medium heat. [step] You can use multiple a cup (240 milliliters), or you can use the entire pot.\nB. [title] Place the chocolate chips into the saucepan and simmer them for 5 minutes. [step] This will create a process called \" processing, \" which means that the chocolate chips will break apart if they are not melted quickly enough.\nC. [substeps] If you are using a bowl, make sure that it fits comfortably over the saucepan and that there are no gaps where heat can escape. Double boilers are great if you need the chocolate to stay melted for a long time (ie: when you are making chocolate dipped strawberries).\nD. [substeps] The heat of the water will help melt the chocolate chips. Make sure that the temperature of the water is roughly nine to 12 degrees fahrenheit (4 to 5 degrees celsius).","prompt":"[header] How to melt chocolate chips [title] Fill a saucepan with 2 inches (5.08 centimeters) of water. [step] You can use an actual double boiler, or improvise your own with a saucepan and a heat-safe bowl. Do not place the second saucepan or bowl on top just yet, however.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6683,"error":null,"output":"C"}
{"question_id":"hellaswag:00001236","item_index":275,"attempt":0,"prompt_hash":"07e26575407aecce501676852ed7ff54fc7aa0fae769678e72dadc3fecdda205","question":"The girl is on front a sink and puts soap on her face, then she remembers the two girls splashing water on her face. then\n\nA. she dons a wrong shoe and falls during her fall on the floor.\nB. she goes to the kitchen area in some small fenced in area.\nC. , the girl is washing the hands under the hot boiling water, then she wash the stomach and the hands together.\nD. , the girl takes water with both hands and splash on her face and some water spills on the floor.","prompt":"The girl is on front a sink and puts soap on her face, then she remembers the two girls splashing water on her face. then","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6683,"error":null,"output":"D"}
{"question_id":"hellaswag:00005195","item_index":276,"attempt":0,"prompt_hash":"588ae96221329075ef294ce30e2ab95c2165c6522938dd4a56345bbc25ff7d9a","question":"[header] How to make crispy cream cheese wontons [title] Prepare the filling. [step] Take ½ cup of cream cheese and let it sit out in room temperature for around thirty minutes to soften it a bit. Remove the end and top part off one green onion.\n\nA. If you are using green onion, you can skip this step; the whole wonton should be pulled out. [title] Separate and combine the eggs and softened cream cheese together.\nB. Toast the peeled onion for 5 minutes at a low temperature. With this mix mix, cook the wontons until they're soft.\nC. If the green onion is a thick one, slice the thick stem in two. Now, finely mince the green onion and add ¼ cup of the minced green onion to the cream cheese.\nD. Using a knife or spoon, thoroughly cut the onion into coarse-looking pieces. Allow the pieces to cool and then transfer them to a large bowl.","prompt":"[header] How to make crispy cream cheese wontons [title] Prepare the filling. [step] Take ½ cup of cream cheese and let it sit out in room temperature for around thirty minutes to soften it a bit. Remove the end and top part off one green onion.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6680,"error":null,"output":"C"}
{"question_id":"hellaswag:00005152","item_index":277,"attempt":0,"prompt_hash":"aa288d9202761707387dce569fd05c42714f33750f4afe88dc91095bd2f25d3b","question":"[header] How to tell if your vans shoes are fake [title] Run the barcode. [step] The box should have a label that includes shoe size, country of production, and a barcode. Use your phone to scan the barcode.\n\nA. [substeps] Make sure that the barcode is for your vans shoes. Even if the barcode indicates that the vans are fake, the barcode will never contain any u.s.\nB. If it's not attached to the label, do not run a search or look at the real box either. [title] Check the height of the barcode.\nC. The barcode should correspond to the type of shoe in the box. [substeps] To scan a barcode with your phone, go to the app store that corresponds with your phone.\nD. If the barcode appears in the copy somewhere where people could expect it, then it is real. There are three major kinds of fake vans.","prompt":"[header] How to tell if your vans shoes are fake [title] Run the barcode. [step] The box should have a label that includes shoe size, country of production, and a barcode. Use your phone to scan the barcode.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6683,"error":null,"output":"C"}
{"question_id":"hellaswag:00005160","item_index":278,"attempt":0,"prompt_hash":"1d3f1ee62df857985489a117744b27926649dd877165fca430c92af3ffe485df","question":"[header] How to dye hair bright red under black hair [title] Avoid washing your hair 24 to 48 hours before dyeing it. [step] Your hair may feel a little greasy beforehand, but try to tough it out. The natural oils present in unwashed hair will help to protect your scalp and roots from the hair dye, especially when you get to the red layer.\n\nA. This means you should wash your hair every 2-3 days. [substeps] If you need to wash your hair at work or at home, it is best to do so 2-3 times a week in the shower.\nB. The hair dye will also penetrate your hair better and the results will be more vibrant if your hair isn't squeaky clean when you color it. [substeps] If you must wash your hair before you dye it, use shampoo only and skip the conditioner.\nC. In between washes, use a low-intensity hair cream. On greasy hair, the worst thing you could do would be brush it out.\nD. [title] Try a mustard mask once a week. [step] To help wash your hair and rinse it out, try endyne \" mustard masks \" that contain a strip of gelatin that stays in the hair overnight.","prompt":"[header] How to dye hair bright red under black hair [title] Avoid washing your hair 24 to 48 hours before dyeing it. [step] Your hair may feel a little greasy beforehand, but try to tough it out. The natural oils present in unwashed hair will help to protect your scalp and roots from the hair dye, especially when you get to the red layer.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6681,"error":null,"output":"B"}
{"question_id":"hellaswag:00003081","item_index":279,"attempt":0,"prompt_hash":"833dc21bc3a58dce20902d52b5d13a0449354edc117d9ba95975fa6038ca1b5d","question":"A woman stands in fronts of an orchestra holding a violin. The orchestra begins to play. the woman\n\nA. smokes a cigarette.\nB. takes the violin in her hands.\nC. stops playing and turns around.\nD. starts playing her violin.","prompt":"A woman stands in fronts of an orchestra holding a violin. The orchestra begins to play. the woman","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6681,"error":null,"output":"D"}
{"question_id":"hellaswag:00005899","item_index":280,"attempt":0,"prompt_hash":"c1ba853766c85f95dc51515e4ab3dfb76796e332ba3ce1c2e2c96fc5874fd183","question":"[header] How to clean a fish bowl [title] Figure out how much water your bowl holds. [step] In order to determine how much clean water you'll need to prepare, you first need to gently pour the bowl's current water and the fish into a clean, soap-free bucket or container. Then, mark the water level on the outside of the bucket.\n\nA. This will give you specific guidelines about how much water each container should hold. Fill a bucket or container with warm, soapy water and set it underneath your bowl.\nB. [substeps] Only use water that is sterilized. The amount of clean water your bucket holds will vary with the amount of fish you intend to clean.\nC. [title] Figure out how much new water you'll need. [step] When you clean your fish bowl, you'll need to replace 40-50% of the total volume of water that it holds.\nD. Repeat this process for each gallon or liter of water that you use. [substeps] Be sure to record this number for easy reference.","prompt":"[header] How to clean a fish bowl [title] Figure out how much water your bowl holds. [step] In order to determine how much clean water you'll need to prepare, you first need to gently pour the bowl's current water and the fish into a clean, soap-free bucket or container. Then, mark the water level on the outside of the bucket.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6682,"error":null,"output":"C"}
{"question_id":"hellaswag:00003104","item_index":281,"attempt":0,"prompt_hash":"5af1b0386b5b14d580f861c5d498b958b96d35c8bc403da225a53338f20b99b4","question":"A large chandelier sparkles over a stage. People in black clothing appear, backs turned. a woman\n\nA. makes a gymnastic flip on the stage.\nB. taps a stick on a violin.\nC. lifts a crown, then the man starts dancing.\nD. climbs onto large, black beams.","prompt":"A large chandelier sparkles over a stage. People in black clothing appear, backs turned. a woman","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6681,"error":null,"output":"A"}
{"question_id":"hellaswag:00001532","item_index":282,"attempt":0,"prompt_hash":"e2281e61ee33f6025691f787981f86f281422341643060fc134ee860c406c8d8","question":"We see team of boaters in a competition in the sea. We see the race start and the boaters take off. They pass a small boat in the water. we\n\nA. see sea life in a reef.\nB. see one team up close.\nC. watch as the boaters perform handstands in the river.\nD. see the ending title screen on black.","prompt":"We see team of boaters in a competition in the sea. We see the race start and the boaters take off. They pass a small boat in the water. we","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6681,"error":null,"output":"D"}
{"question_id":"hellaswag:00003939","item_index":283,"attempt":0,"prompt_hash":"3f6ac4ebf56847a112769b483b4024e9e5b64f5c14e6eb2c2780c8a3cef03134","question":"[header] How to remedy hemorrhoids with simple folk medicine [title] Purchase witch hazel. [step] You can find it at the store in the liquid astringent form (typically in 12 ounce bottles) and a tub of baby wipes, but do not toss it in a toilet. It's best to use wipes that are toilet safe by charmin or other companies.\n\nA. [substeps] Witch hazel alone does not correct hemorrhoids, so read the packaging carefully. If you don't like the way it smells, put some witch hazel into your toilet twice a day.\nB. Some refills also contain a new tub. [title] Add the witch hazel to the wipes.\nC. [substeps] Wash your hands first with hand sanitizer before using witch hazel, and squeeze the alcohol back into the bottle. Add ½ tablespoon (30 ml) of witch hazel to half of a glass of milk.\nD. [title] Wash your hands with warm water and soap. [step] Pour witch hazel from the distilled container over your hands and then run the water over your hands, scrubbing your hands.","prompt":"[header] How to remedy hemorrhoids with simple folk medicine [title] Purchase witch hazel. [step] You can find it at the store in the liquid astringent form (typically in 12 ounce bottles) and a tub of baby wipes, but do not toss it in a toilet. It's best to use wipes that are toilet safe by charmin or other companies.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6886,"error":null,"output":"D"}
{"question_id":"hellaswag:00006345","item_index":284,"attempt":0,"prompt_hash":"7338f8ccc99ef6ee2e49ceae2163ff43c0ed62b5d0ca7d14e1a3e53b34b2cc66","question":"[header] How to calculate the standard deviation of a portfolio [title] Calculate the standard deviation of each security in the portfolio. [step] First we need to calculate the standard deviation of each security in the portfolio. You can use a calculator or the excel function to calculate that.\n\nA. Then to calculate the standard deviation of each security's stock, we will need to calculate that, too. [substeps] To calculate the standard deviation of a stock that is less secure, we should multiply the standard deviation by the stock price to calculate the standard deviation.\nB. Be sure that you have the correct gv value for each security, as well as the same symbol depending on the portfolio. [substeps] If you have the right condition of the security, then the standard deviation is adjusted to reflect that condition.\nC. [substeps] Let's say there are 2 securities in the portfolio whose standard deviations are 10% and 15 %. [title] Determine the weights of securities in the portfolio.\nD. Once you have found the standard deviation of a portfolio, we then trace it down. As we did the example in step 4, the standard deviation of 24 security in the portfolio would say \" 52 \".","prompt":"[header] How to calculate the standard deviation of a portfolio [title] Calculate the standard deviation of each security in the portfolio. [step] First we need to calculate the standard deviation of each security in the portfolio. You can use a calculator or the excel function to calculate that.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6887,"error":null,"output":"C"}
{"question_id":"hellaswag:00008583","item_index":285,"attempt":0,"prompt_hash":"10cd6674246fcaefedc42e8ffd2b3483402219d5c48ece89267ad3ecadaafa3c","question":"[header] How to make waxed dipped bears [title] Place 1 pound of wax in a slow cooker. [step] Set the temperature of the cooker on low to medium. Use a cooking thermometer to measure the temperature until it reaches about 180 degrees fahrenheit (82 c).\n\nA. [substeps] It's best to place the wax in a cupboard or other long-handled cooler. Bear wax is flammable and can melt the blades of a thermometer.\nB. Pour the wax into a slow cooker. [substeps] If you don't have an electric slow cooker, wrap the wax in a wax ball, tuck in the wax ball, and bake it.\nC. This temperature should warm the waxed milk completely, thereby melting the wax. [substeps] The wax should be a bit hard in your hands.\nD. One pound of wax will typically cover 2 small bears or 1 medium-sized bear. [substeps] Add glitter to the wax before dipping the bear if you want it to sparkle.","prompt":"[header] How to make waxed dipped bears [title] Place 1 pound of wax in a slow cooker. [step] Set the temperature of the cooker on low to medium. Use a cooking thermometer to measure the temperature until it reaches about 180 degrees fahrenheit (82 c).","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6886,"error":null,"output":"D"}
{"question_id":"hellaswag:00003558","item_index":286,"attempt":0,"prompt_hash":"c54f1ef4bc35f95935a3084499a79813806619438dd0ae35393261042c9e780d","question":"[header] How to lighten hair at home [title] Rinsing hair with lemon juice is one of the most convenient ways to lighten it at home. [step] Lemon juice is full of citric acids, which have a bleaching effect when they are repeatedly applied to hair. [substeps] You will most likely need to repeat this process several times to see a significant change in your hair color.\n\nA. If you have several hours to mix lemon juice into your hair and sit out in the sun, consider the methods listed in this article: use lemon juice to lighten hair. If you are looking for a quicker solution, read on.\nB. It may take several days, but after a few weeks, you will be glad you didn't blow-dry your hair. [title] Skip the shampoo and conditioner.\nC. [title] Use a despirat or dishwater solution from the supermarket. [step] These are mild acids which moisturize hair very easily.\nD. [title] Use one half ounce lemon juice and one tablespoon lime juice. [step] Lemon juice contains citric acids that have a bleaching effect when applied to hair.","prompt":"[header] How to lighten hair at home [title] Rinsing hair with lemon juice is one of the most convenient ways to lighten it at home. [step] Lemon juice is full of citric acids, which have a bleaching effect when they are repeatedly applied to hair. [substeps] You will most likely need to repeat this process several times to see a significant change in your hair color.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6886,"error":null,"output":"D"}
{"question_id":"hellaswag:00000448","item_index":287,"attempt":0,"prompt_hash":"67a3645e972ad4660c4411935ff9c0994225665bb35eda40693761d482bbf4ba","question":"A man is on a roof cutting throwing shingles. The man removes the roofing. the man\n\nA. then installs a new roof.\nB. throws the shingles at the roof to clean it.\nC. removes the rubber tire valve knob steel and reinserts it in the roof slowly.\nD. raises an axe from a hole in the roof.","prompt":"A man is on a roof cutting throwing shingles. The man removes the roofing. the man","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6886,"error":null,"output":"B"}
{"question_id":"hellaswag:00002227","item_index":288,"attempt":0,"prompt_hash":"ee99043f6bab00c02518909566559402ea5e96a88df31cfbccb4aa5f616032cb","question":"The woman is shown putting her feet into her shoes while they are tied. but\n\nA. the laces are removed one at a time, and the woman laces up the other end of the laces and moves herself a few inches to adjust it.\nB. after a few minutes of her lifting her feet off of the ground, she takes them off without using her hands.\nC. as she puts them on she pretty much falls down on the ground and they put her shoes on her feet.\nD. still ends up facing wrong direction.","prompt":"The woman is shown putting her feet into her shoes while they are tied. but","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6900,"error":null,"output":"B"}
{"question_id":"hellaswag:00005140","item_index":289,"attempt":0,"prompt_hash":"48a54fc61d3ac9a004777e0d4ee6d57431add823fa358d2898325dc8ea5fd9e5","question":"[header] How to be a successful pilgrim [title] Feel as if god wants you to undertake this journey and remain eager to see the temple. [step] Slowly, bring the shrine and the particular deity into your consciousness, right from the time you get the idea to go on the trip. Search the net for articles and experiences of people who visited the same place and your urge to make it happen will become stronger.\n\nA. As that happens, the universe will pave the way for you to reach your destination, and all the details will fall into place. [title] Book a flight after checking the timings of the place of worship.\nB. Go all over the world and look around. Give thanks and recognition to the divine, and unto god for your journey.\nC. [substeps] For more information on making a crucial pilgrimage with an unusual deity, see. [title] Accept your presence at the temple and be willing to welcome the people who go there with you.\nD. Try to do all you can to be happy and hopeful whilst standing confidently to the end of your journey. It's the dream, not a hope! [title] Find the halls of worship.","prompt":"[header] How to be a successful pilgrim [title] Feel as if god wants you to undertake this journey and remain eager to see the temple. [step] Slowly, bring the shrine and the particular deity into your consciousness, right from the time you get the idea to go on the trip. Search the net for articles and experiences of people who visited the same place and your urge to make it happen will become stronger.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6895,"error":null,"output":"A"}
{"question_id":"hellaswag:00000492","item_index":290,"attempt":0,"prompt_hash":"9827a44fc971bce99aea78f7c9115cdd54cfa3b3ede70857b90e8d28502c3ddd","question":"A person is wake boarding behind a boat. they\n\nA. go back and forth over the waves of the boat.\nB. are at the watership, then they cables up into the system and rests.\nC. are swimming behind a boat.\nD. lower a hose into a cup.","prompt":"A person is wake boarding behind a boat. they","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6897,"error":null,"output":"A"}
{"question_id":"hellaswag:00001456","item_index":291,"attempt":0,"prompt_hash":"ea80e7cd7dd3c56951a4798f1fb841d8d6466320cc2b879b1bbb907b3c8fbbd3","question":"A man in a yard starts his lawn mower. the man\n\nA. starts mowing the lawn with a red paint sprayer.\nB. starts mowing the lawn in a racer and someone else starts mowing the lawn behind him.\nC. mows his lawn with headphones on his ears as his dog walks around the yard.\nD. slowly begins mowing the yard using the remote control.","prompt":"A man in a yard starts his lawn mower. the man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6897,"error":null,"output":"C"}
{"question_id":"hellaswag:00003648","item_index":292,"attempt":0,"prompt_hash":"37d654f1f3363ebc6d44a8963e11d389b08ac6fd46e9f1727ffc6743cf66e06a","question":"[header] How to buy an assault rifle [title] Identify the assault rifle you want. [step] A modern assault rifle can shoot up to 300 meters and has selective firing mechanisms. Common assault rifles include the ak-47, m4a1 carbine, and m16.\n\nA. M4a1's are made of a very thick plastic. Sp metal (a.k.a.\nB. You'll have to study both to find the right one to suit your needs and experience. [title] Determine which weapon you want to carry.\nC. Also think about extra features that you might want with the assault rifle : [substeps] Sights electronics collapsible stocks suppressors lasers drum magazines [title] Ask an expert. [step] If you don't know what rifle you want, then try speaking to someone who is familiar with the different assault rifles on the market.\nD. [substeps] A soldier: recon (190). A medic: sniper/doc (190).","prompt":"[header] How to buy an assault rifle [title] Identify the assault rifle you want. [step] A modern assault rifle can shoot up to 300 meters and has selective firing mechanisms. Common assault rifles include the ak-47, m4a1 carbine, and m16.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6895,"error":null,"output":"C"}
{"question_id":"hellaswag:00004565","item_index":293,"attempt":0,"prompt_hash":"6e436f701f96db772f1f5e89c336b202b18af7e76287907f204e2693799674d0","question":"[header] How to wear pink [title] Wear pink with more neutral colors. [step] Pink looks great, but it can sometimes be difficult to pair it with other colors. Choosing one pink statement piece, such as a shirt, a nice pair of pink pants, or even a pink jacket will look great if you pair it with a more neutral color like white, black, or khaki.\n\nA. [substeps] Pink has the added benefit of being bold, as you can mix and match colors. You can wear pink with dress shoes, but with a pink tank top and solid purple jeans, it's almost impossible to completely match shades.\nB. [substeps] Opt for neutral colors like brown, grey, dark blue, or light turquoise. These are colors that will also work, since they don't clash too much with the shade of pink in your outfit.\nC. [substeps] Pink shirts and jackets, on the other hand, can give an even more interesting appearance. Try pairing pink with more neutral, floral colors, like a tank top or a purple jean top.\nD. [substeps] For example, wearing a light pink jacket with a white dress or skirt will look mature and elegant. If you wear a pink outfit, try accessorizing with different colors.","prompt":"[header] How to wear pink [title] Wear pink with more neutral colors. [step] Pink looks great, but it can sometimes be difficult to pair it with other colors. Choosing one pink statement piece, such as a shirt, a nice pair of pink pants, or even a pink jacket will look great if you pair it with a more neutral color like white, black, or khaki.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6895,"error":null,"output":"D"}
{"question_id":"hellaswag:00001991","item_index":294,"attempt":0,"prompt_hash":"96dad93f85ece40b242a07e2893f0d39479868cbb623e133da9bd4caa9f31415","question":"Woman is posing to a camera in a white room. man\n\nA. is taking off his shirt showing a tattoo and taking out the needles.\nB. is standing behind the wmoan cutting her hair with a scissor and the model is posing.\nC. is standing in a room on stilts doling out tattoos.\nD. is standing in a shower ina white room showing how to wash back the white clothes in front of the mirror.","prompt":"Woman is posing to a camera in a white room. man","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6895,"error":null,"output":"B"}
{"question_id":"hellaswag:00005191","item_index":295,"attempt":0,"prompt_hash":"ec3e057ddb2cf881ff7814029baf2cfcd00c5105a527b21523e0f99d1f7b2eef","question":"[header] How to naturally dye your hair [title] Try lemon. [step] If you want to lighten your hair naturally, try using lemon. Lemon may be able to lighten your hair a shade or 2.\n\nA. This is especially helpful for hair that is dry and brittle. [substeps] Mix half a cup of mild lemon juice with a gallon of water.\nB. You can even make lemon with lemon-infused coconut oil. Don't dilute coconut oil in an avocado oil mixture.\nC. [substeps] Mix a cup of water with two tablespoons of lemon juice. Then, apply it to wet hair and allow your hair to dry naturally.\nD. [substeps] Lemon juice contains vitamins a, e, and k that add color to hair and lighten it. Lemon also helps lighten hair, so you want to drink more than 1 vitamin a with lemon juice if you have very dark hair.","prompt":"[header] How to naturally dye your hair [title] Try lemon. [step] If you want to lighten your hair naturally, try using lemon. Lemon may be able to lighten your hair a shade or 2.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6897,"error":null,"output":"C"}
{"question_id":"hellaswag:00004200","item_index":296,"attempt":0,"prompt_hash":"1c956e0c9455c3e2677ddf0da9f2bcf91da7f85ee1ed7a1e98ef0ce85e211268","question":"[header] How to apply everyday makeup [title] Start out with a completely clean face. [step] The very first part of your routine is to clean your face. Use a washcloth, soap, and warm (but not hot) water to gently wash your face.\n\nA. If you shower in the morning and already scrubbed your face, you can of course skip this step. [substeps] It's even more important to make sure your face is clean at the end of the day.\nB. This is important to remove any traces of dirt, then scrub and clean yourself. [substeps] You don't need to use a washcloth.\nC. Be careful when using shampoo or a moisturizer. You should also be very gentle around the eyes, while using a face spray with salicylic acid.\nD. [substeps] Make sure to do so in the morning if you normally wash your face after every meal. You can apply fresh face wash in the morning if you want to but it's okay to skip this step if it doesn't pair well with your normal everyday routine.","prompt":"[header] How to apply everyday makeup [title] Start out with a completely clean face. [step] The very first part of your routine is to clean your face. Use a washcloth, soap, and warm (but not hot) water to gently wash your face.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6894,"error":null,"output":"A"}
{"question_id":"hellaswag:00009861","item_index":297,"attempt":0,"prompt_hash":"4b9244ac56d89ae1929eab45132e1ade19bb06e78308fa68a26ea34064f15dd4","question":"[header] How to keep a dress shirt from riding up [title] Tuck your shirt in like normal. [step] Starting with your pants unbuttoned, push the tail of the shirt down into your waistband. Make sure it's straight and fully extended so there are no wrinkles.\n\nA. Secure the waist of the shirt using belt loops or other pull-down accessories. [title] Secure the front of the shirt with straight pins.\nB. You want to start with as crisp and clean a foundation as possible. [substeps] The military tuck will work best with a shirt that reaches at least 3-4 inches (7.6-10.2 cm) below your waistband.\nC. Tighten the sleeves as you go so that they're snug. You'll need to be able to adjust the shirt as needed to keep a snug fit when standing straight and tall.\nD. Pin it down on the shoulder straps so it goes all the way up to the collar, and close the button right above the collar. [title] Insert a handkerchief into your dress shirt.","prompt":"[header] How to keep a dress shirt from riding up [title] Tuck your shirt in like normal. [step] Starting with your pants unbuttoned, push the tail of the shirt down into your waistband. Make sure it's straight and fully extended so there are no wrinkles.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6895,"error":null,"output":"B"}
{"question_id":"hellaswag:00008526","item_index":298,"attempt":0,"prompt_hash":"34ab1eacda1ff96e515055ffdddc0f266e8fdddc19efff0b18b6f4fcbaa0b039","question":"[header] How to insert a video link in constant contact [title] Visit the constantcontact website. [title] Click the \" log in \" button in the top right hand corner of the page. [title] Enter the user name and password associated with your constant contact account and click \" login \".\n\nA. [step] You will be taken immediately to the \" home \" tab of your account page. [title] Select the edit icon next to the email you want to insert a video link in.\nB. [title] Follow the on-screen prompts to complete the blank page and link your video to the website. [title] Click the \" continue \" button to continue to the page.\nC. [title] Click the \" friends \" dropdown list. [step] If you don't have any friends on constantcontact, you will have to make another visit.\nD. [title] Click the \" deactivate \" button at the top right hand corner of the page. [step] As with the other website applications, such as google adisco.com, you are asked to confirm that you want to deactivate your web account.","prompt":"[header] How to insert a video link in constant contact [title] Visit the constantcontact website. [title] Click the \" log in \" button in the top right hand corner of the page. [title] Enter the user name and password associated with your constant contact account and click \" login \".","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6893,"error":null,"output":"A"}
{"question_id":"hellaswag:00005257","item_index":299,"attempt":0,"prompt_hash":"05aaf0b5dd2e764f8e3bc61c0018aee7a566e8701fed2712ca00670d2470d27a","question":"[header] How to fake bangs [title] Start with a center part and neatly-brushed hair. [step] This style works with most lengths of hair. It will be even more convincing if your hair is long enough to tuck behind your ears, however.\n\nA. [substeps] You can create a neat-looking center part on 1 side of your hair, or secure it with a clip. [title] Bring the base of your hair up.\nB. Otherwise, your bangs will look a bit cartoonish and unsightly in a standard style. [substeps] You can find equally good quality hairspray designed for your hair at most drugstores.\nC. [substeps] If you want the illusion of side bangs, part your hair on the side. [title] Separate the front-left section of your hair from the rest.\nD. This is also a good step-up from slightly greasy hairstyles to more evenly-textured options. [substeps] For medium or fine hair, consider it a less drastic style.","prompt":"[header] How to fake bangs [title] Start with a center part and neatly-brushed hair. [step] This style works with most lengths of hair. It will be even more convincing if your hair is long enough to tuck behind your ears, however.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6894,"error":null,"output":"C"}
{"question_id":"hellaswag:00004101","item_index":300,"attempt":0,"prompt_hash":"15ea6733cb6bd0aad3c0d92abb3f7eb36e0b1d0d5a465ed5ac021cba2b55250b","question":"[header] How to soak clothes [title] Pre-soak your clothes before washing. [step] You can soak clothes directly in the chamber of your washing machine if you plan to machine wash them afterward. All you need to do is add detergent to the water drawn into the washing machine, then let the clothes soak for 20-30 minutes in the mix of detergent and standing water.\n\nA. For the fastest results, fill a colander with water and set it next to a sink in your washing machine. Fill the washing machine with the hottest water you can stand before submerging the clothes.\nB. [substeps] If you're putting your clothes in the machine with more water, they could be covered with dirty clothing later. [title] Wash your clothing before putting them in the machine.\nC. [substeps] It will be easier to soak clothes in a top-loading washing machine than in a side-loading machine. Check your side-loading machine for an integrated pre-soak function.\nD. [substeps] Soak an old rag or sponge in warm water for around 10 minutes. Turn off the water once the fabric is fully soaked with the solution.","prompt":"[header] How to soak clothes [title] Pre-soak your clothes before washing. [step] You can soak clothes directly in the chamber of your washing machine if you plan to machine wash them afterward. All you need to do is add detergent to the water drawn into the washing machine, then let the clothes soak for 20-30 minutes in the mix of detergent and standing water.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6895,"error":null,"output":"C"}
{"question_id":"hellaswag:00004006","item_index":301,"attempt":0,"prompt_hash":"e58020e84af2a3dcb5bfcbc3076ca4f49c5394e26990a5160d6c7e4244d7d4c2","question":"[header] How to include spanking in child discipline [title] Find a private area. [step] Conducting this type of discipline in a private area preserves the dignity of your child and prevents unnecessary embarrassment. The focus should be on discipline and additional embarrassment for your child should be minimized.\n\nA. [substeps] Look for a private area free of distractions. Have a book or other non-distracting object nearby to quiet the child and yourself.\nB. [substeps] Most experts are adamant that children should never be spanked under any circumstances. Nonetheless, some parents believe that spanking is the best way to get children to adhere to rules.\nC. [substeps] If you have permission to do so, choose a private area to discipline your child in private. It will help your child feel at home instead of surrounded by other adults.\nD. There are a wide variety of private areas you can think about according to age and the individual or group of children that you're spanking. [substeps] Children should have close supervision at all times.","prompt":"[header] How to include spanking in child discipline [title] Find a private area. [step] Conducting this type of discipline in a private area preserves the dignity of your child and prevents unnecessary embarrassment. The focus should be on discipline and additional embarrassment for your child should be minimized.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6895,"error":null,"output":"B"}
{"question_id":"hellaswag:00004140","item_index":302,"attempt":0,"prompt_hash":"d8e9bea07a0e94dd60ec79e34c6789671d8df087946cadbab968f45ab0a49775","question":"[header] How to avoid getting sick at school [title] Wash your hands as often as possible. [step] Wash your hands thoroughly with warm water and soap before you eat, and after you use the restroom or come into contact with anything that other students have used before you. Washing your hands often may cause dry skin, so carry moisturizing lotion during winter months.\n\nA. [substeps] First wet your hands with warm or cool running water, and add soap. The water doesn't need to be hot to clean germs and microbes from your hands.\nB. [substeps] If your school allows hand washing, then take care to do so after a meal. If you make a mess in the bathroom, then you are much more likely to get sick at the end of the day.\nC. [substeps] Call ahead to inform teachers of your sickness by phone or text message. Otherwise, bring a copy of your school paper with you to school.\nD. [substeps] While it's convenient for you to use the bathroom on your own, you may want to substitute soap when you're traveling. Some people use sanitary hand sanitizers for the bathroom.","prompt":"[header] How to avoid getting sick at school [title] Wash your hands as often as possible. [step] Wash your hands thoroughly with warm water and soap before you eat, and after you use the restroom or come into contact with anything that other students have used before you. Washing your hands often may cause dry skin, so carry moisturizing lotion during winter months.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6896,"error":null,"output":"A"}
{"question_id":"hellaswag:00008164","item_index":303,"attempt":0,"prompt_hash":"347ee080a02b3a78683279873e53757f1e552d2adcd064f27a5afe8d590f918e","question":"[header] How to shower after gym class while on your period [title] Know the rules at your school. [step] In some cases you may be required to shower after gym class, but in other cases it may be optional. Find out what the policy is at your school and whether they are willing to make exceptions.\n\nA. [title] Check with your school's gym clothes department if school policies have been violated. [step] In some cases you may be expected to wear loose clothing while you are on your period, but if you are allowed to be in your underwear at all times during classes, this might be a way of breaking the rules.\nB. [substeps] This may depend on the rules of your school. Usually, schools will only allow you to shower with your period if you are physically fit and look clean.\nC. [substeps] If they don't let you shower after gym class, you may never be allowed to take showers unless you have a new tampons or have been distracted by a new case of infection during class. During regular class periods, look for the rules in your school newspaper or an online forum that lists the rules.\nD. [substeps] Be sure to consider that you may be sweaty and smelly if you choose not to shower after gym class. Choosing not to shower works best if the activities during class do not cause you to sweat much or if you have gym at the end of the day.","prompt":"[header] How to shower after gym class while on your period [title] Know the rules at your school. [step] In some cases you may be required to shower after gym class, but in other cases it may be optional. Find out what the policy is at your school and whether they are willing to make exceptions.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6895,"error":null,"output":"D"}
{"question_id":"hellaswag:00008492","item_index":304,"attempt":0,"prompt_hash":"d90c6e765cab7e5bdce96e46a240ab9685012b0af611d5a34a01a99139f907c2","question":"[header] How to be yourself and impress others [title] Create yourself! [step] You may not know who you are, what you like, what you want to do in life, but this is okay. Create yourself, don't find yourself, create yourself. Create your own personality, but be yourself.\n\nA. Creating your personality doesn't mean copying others, but you can always get influence. Have confidence, a good sense of style and a contagious smile.\nB. Start off trying to feel confident, a natural quality that may help you succeed in work. Your self confidence is what makes you who you are.\nC. Act as a part of an abacus, don't resist being yourself. [substeps] For example, if you love traveling, always go to your destination, don't just say, \" i have a lot of fun traveling.\nD. Don't be sloppy and make you look pale. You don't have to be perfect, you just have to be yourself.","prompt":"[header] How to be yourself and impress others [title] Create yourself! [step] You may not know who you are, what you like, what you want to do in life, but this is okay. Create yourself, don't find yourself, create yourself. Create your own personality, but be yourself.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5545,"error":null,"output":"D"}
{"question_id":"hellaswag:00008357","item_index":305,"attempt":0,"prompt_hash":"d0a515e0206536c2722477807c4cc9326888ce0188f07b93a95ed805a882b9ab","question":"[header] How to connect your nintendo wii to the internet [title] Ensure that your network is properly set up. [step] You will need to be properly broadcasting a signal in order to connect the wii to the network. Refer to your router or modem's instructions in order to properly set up your network.\n\nA. [title] Remove and disconnect the wii remote cable. [step] This is the same cable that came with the wii remote.\nB. [title] Connect the wii to your network using ethernet cable. [step] Most wii's have an ethernet cable available for purchase.\nC. [substeps] If you can connect to your network with other wireless devices, then you should have no issues connecting your wii. No special adjustments need to be made to the wireless network to accommodate it.\nD. [title] Check your router's operating system. [step] Your nintendo may have a remote power-off switch that will turn off the power if not installed.","prompt":"[header] How to connect your nintendo wii to the internet [title] Ensure that your network is properly set up. [step] You will need to be properly broadcasting a signal in order to connect the wii to the network. Refer to your router or modem's instructions in order to properly set up your network.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5545,"error":null,"output":"B"}
{"question_id":"hellaswag:00001422","item_index":306,"attempt":0,"prompt_hash":"9ef79a685dad4c5b5dc6cfd45b863f4de791a56a6a882eaa4324862ae6bd3698","question":"A woman is adjusting a camera in a bedroom. She is smoking a cigarette and ashing it into a tray. she\n\nA. puts it back in place and spins it around again.\nB. blows out the smoke and shows it off in the end.\nC. continues smoking from the tray.\nD. grinds the cigarette into the tray when she is finished.","prompt":"A woman is adjusting a camera in a bedroom. She is smoking a cigarette and ashing it into a tray. she","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5547,"error":null,"output":"D"}
{"question_id":"hellaswag:00001305","item_index":307,"attempt":0,"prompt_hash":"b82de78f5a3764cd4367f367aac586f312e10f6ff02e5f2b6669b345e4e512f8","question":"A man is standing with his legs apart and is holding a sprayer in his right hand and spraying a dresser with many drawers as a man walks past him and looks at what he's doing. The man brings his legs closer together while still spraying and he begins closing the drawers. the man\n\nA. stops and looks at the camera and is now getting excited.\nB. then begins bending all over and showing off his back collar.\nC. turns his body towards the drawer and is shaving in front of him while shaving with the sprayer.\nD. then goes to the right side of the dresser and sprays there, then goes back to the front, then the top, and then on the left side.","prompt":"A man is standing with his legs apart and is holding a sprayer in his right hand and spraying a dresser with many drawers as a man walks past him and looks at what he's doing. The man brings his legs closer together while still spraying and he begins closing the drawers. the man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5547,"error":null,"output":"D"}
{"question_id":"hellaswag:00004384","item_index":308,"attempt":0,"prompt_hash":"b256d0aa7cff8f215dc086c3417a8b106a561f0e99ee9129e96cbbf9289b2e85","question":"[header] How to prevent constipation in toddlers [title] Hydrate your child. [step] Offer your child at least 32 ounces of water or diluted juice every day. Try to make 75% of that water, since juice contains sugar that can cause diarrhea.\n\nA. This extra fluid in the stool will make it softer and easier to pass. [substeps] To dilute juice, add 1 ounce of water for every ounce of juice.\nB. Boys should drink 27 ounces of water everyday, and girls should drink 7 ounces of water three times daily. [substeps] In between meals, try giving your child an apple or a lemon.\nC. Don't add more water if you are using tea or juice that has sugar. [substeps] Make sure you avoid regularly taking juice before a meal and stay hydrated throughout the day.\nD. If needed, take your child to the bathroom (or another bathroom office) in general. Change your water or juice out if it's right before bed at night.","prompt":"[header] How to prevent constipation in toddlers [title] Hydrate your child. [step] Offer your child at least 32 ounces of water or diluted juice every day. Try to make 75% of that water, since juice contains sugar that can cause diarrhea.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5545,"error":null,"output":"A"}
{"question_id":"hellaswag:00000978","item_index":309,"attempt":0,"prompt_hash":"1f2bca29488af0366e7d615624021dad7bacdeaf3ed7b2912122599a4263eb54","question":"Finally he uses a long metal hook to hook the fish he caught and brings it out the hole. he\n\nA. tightens the hook on the fish to catch it again.\nB. pulls a red needle out of the hole and begins going back in to try again trying and pulling some more fish out.\nC. then unhooks the long hook and use his tools to unhook the small fishing hook the the fish mouth leaving the fish bloody.\nD. fish out of the hole and attempts to catch some more.","prompt":"Finally he uses a long metal hook to hook the fish he caught and brings it out the hole. he","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5546,"error":null,"output":"A"}
{"question_id":"hellaswag:00007179","item_index":310,"attempt":0,"prompt_hash":"58c10fa32e225e7b3171d76a2dc7da9056d8b3e10828f727f798e40244b38642","question":"[header] How to create a wikipedia town [title] Be prepared for a lot of work. [step] This is not something that can be done in a day, week, or even months. [title] Get others invested in it.\n\nA. [step] You will want to get the town councils (in whatever form they are), businesses, chamber of commerce, citizens, etc. For this to be successful, it will need to be a community project.\nB. [step] Be sure to have a large library, a forum, a blog, a website and plenty of fun. Most of the list is going to be written in a large readable format, and can get bumpy at times.\nC. [step] [ now you should have a few people who aren't too careful about their own wikipedia entries and it gives them lots of expansions to share with friends and create articles as well. [title] Have a forum, newsletter, and web page.\nD. [step] Ask your friends or your fellow wikipedia enthusiasts if they have any recommendations about what wikipedia might be like. If they mention it, write your comment! [title] Plan a website, if you already have one.","prompt":"[header] How to create a wikipedia town [title] Be prepared for a lot of work. [step] This is not something that can be done in a day, week, or even months. [title] Get others invested in it.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5544,"error":null,"output":"A"}
{"question_id":"hellaswag:00004895","item_index":311,"attempt":0,"prompt_hash":"7fd81ec50eff8394b42daad359dce293a7d88dec892152531c866b68ffb038be","question":"[header] How to treat epilepsy in beagles [title] Take your dog to the vet. [step] Your vet will do some testing to rule out other possible causes for the seizures. If no cause for the beagle's seizures is identifiable, the dog is assumed to have epilepsy.\n\nA. [substeps] The veterinarian may want to test for poisoning, liver disease, diabetes, kidney disease, and anemia, among other health issues. At this point, treatment depends on the use of anticonvulsant drugs to suppress abnormal electrical activity in the brain, rather than treating the underlying cause.\nB. However, even with ongoing testing, vets are still not ruling out the possibility that something could be wrong with your dog. [substeps] Urine can affect a dog's eating habits.\nC. Your vet may perform the following tests : [substeps] Blood test. The vet may check for tumors on the animal, such as scales.\nD. [substeps] If the vet makes a diagnosis, then they will refer your pup to a canine behavioral specialist for further treatment. [title] Lie down in a comfortable position so that the dog can see you and change direction.","prompt":"[header] How to treat epilepsy in beagles [title] Take your dog to the vet. [step] Your vet will do some testing to rule out other possible causes for the seizures. If no cause for the beagle's seizures is identifiable, the dog is assumed to have epilepsy.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5545,"error":null,"output":"A"}
{"question_id":"hellaswag:00004446","item_index":312,"attempt":0,"prompt_hash":"a921d033c6adef95bb1877c56ee61c03f4bb2eed49e11347083e228b7df5fe79","question":"[header] How to prepare beef ribs [title] Take off the membrane. [step] Ribs have a thick membrane that needs to be removed before cooking-otherwise it gets tough and chewy. Slide your fingers under the membrane and work it off, or use a knife or screwdriver to help it come loose.\n\nA. Discard the membrane once it is removed. [substeps] Be sure to get every last shred of membrane, since you don't want bits of chewy membrane on your ribs.\nB. [substeps] You can also use a fork to remove sebum-a layer of oil that accumulates in the bones after cooking on the skin. [title] Arrange the ribs in a gravy pot.\nC. Be very careful as you work the membrane off the bone. You'll only feel as much if you don't wiggle it off first.\nD. [substeps] You don't have to take off the membrane, especially if you're preparing the ribs in an entree. You can only remove the membrane while cooking on a gas grill or in a roasting pan.","prompt":"[header] How to prepare beef ribs [title] Take off the membrane. [step] Ribs have a thick membrane that needs to be removed before cooking-otherwise it gets tough and chewy. Slide your fingers under the membrane and work it off, or use a knife or screwdriver to help it come loose.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5545,"error":null,"output":"A"}
{"question_id":"hellaswag:00003577","item_index":313,"attempt":0,"prompt_hash":"13a832600e8010f0cbf9334a92aa72b3138a22d336e47d8f6ab394486114501b","question":"[header] How to clean whirlpool tub jets with jetted tub bio cleaner [title] Fill your whirlpool jetted tub with water about 1-2 inches (2.5-5.1 cm) above the highest jet. [step] Point the jets down. [title] Add 1/2 to 1 teaspoon of a jetted tub bio cleaner.\n\nA. [step] Do this slowly to avoid splashing or spilling any of the water. [title] Add a few squirts of your own liquid laundry detergent (not a class 2 but a bit for the sake of efficiency).\nB. [step] This is essential as jetted tubs do not have the same recommended pump cleaning requirements as other ones. [title] Add 1 cup (237 ml) aquarium salt.\nC. [step] With the jetted tub water, use a small tube at the bottom. Fill your filter's tube near the entire nozzle.\nD. [step] This is a gel and will dissolve shortly or when the jets are turned on. [title] Turn on the jets for 10-15 minutes.","prompt":"[header] How to clean whirlpool tub jets with jetted tub bio cleaner [title] Fill your whirlpool jetted tub with water about 1-2 inches (2.5-5.1 cm) above the highest jet. [step] Point the jets down. [title] Add 1/2 to 1 teaspoon of a jetted tub bio cleaner.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5545,"error":null,"output":"D"}
{"question_id":"hellaswag:00009050","item_index":314,"attempt":0,"prompt_hash":"76550871765d54891f5013ab87a11070c7e3b6b06d1f62dba167c97f0f75a8e5","question":"[header] How to prevent static hair [title] Use hairspray. [step] If you live in a dry area and are prone to static hair, hairspray is vital. Using hairspray in the morning and throughout the day can help prevent and eliminate static hair.\n\nA. [substeps] For best results, spray hairspray on your brush. Then, brush your hair while it's still damp.\nB. [substeps] To prevent static, use a bobby pin instead of a regular hairspray. If possible, use this hairspray each day so that it stays static-free throughout the day.\nC. [substeps] Put your hair up in a loose bun and let it air dry. Secure the bun with a hair tie or a sleek t-shirt when you get out of bed.\nD. [substeps] Use hairspray after a shower to help prevent wet hair from sticking to a shower mat. Your hair should now most likely have a static content that you are unable to control.","prompt":"[header] How to prevent static hair [title] Use hairspray. [step] If you live in a dry area and are prone to static hair, hairspray is vital. Using hairspray in the morning and throughout the day can help prevent and eliminate static hair.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5545,"error":null,"output":"A"}
{"question_id":"hellaswag:00004561","item_index":315,"attempt":0,"prompt_hash":"be09a2de759fe49f085f962b92159e5de3a2bb52733804f33dc2908750eadb46","question":"[header] How to deal with visitors after giving birth [title] Explore your feelings about visitors. [step] Chances are that you are just as excited to have your loved ones meet your new baby as they are. But unexpected complications can arise during labor and after that may make welcoming visitors more difficult.\n\nA. Take some time to feel things out and find out what's going on in regards to your baby. [substeps] Even the tiniest hint about a visitor can be startling.\nB. Taking some time to consider whom you'd like to visit and how to deal with potentially difficult situations can help prevent awkward encounters after you give birth. [substeps] Make a list of people whom you'd like as visitors and those who can wait a bit.\nC. Try not to focus on your feelings of anxiety, stress, or worry over having multiple people visit your family. Instead, focus on your attention on how your baby handles visitors.\nD. [substeps] View the birth as a fun day. Visit your child with a sense of pleasure.","prompt":"[header] How to deal with visitors after giving birth [title] Explore your feelings about visitors. [step] Chances are that you are just as excited to have your loved ones meet your new baby as they are. But unexpected complications can arise during labor and after that may make welcoming visitors more difficult.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5824,"error":null,"output":"B"}
{"question_id":"hellaswag:00007772","item_index":316,"attempt":0,"prompt_hash":"2c1187d659e016a36fc1e512b1de194af6b3091ddb29f03c5c68ee9f6fb349db","question":"[header] How to fight off a dog [title] Stay on your feet and shout for help. [step] If you let the dog jump on you or bring you to the ground, you'll be in a much weaker position and more likely to be seriously hurt. If you're in a city or residential neighborhood, call out for the dog's owner.\n\nA. If you're in the woods or a rural environment, you'll need to call out for any help you can get. [title] Place an object between yourself and the attacking dog.\nB. If you are near a phone or an emergency room, call for help from the police. [title] Pull as far away from the dog as possible.\nC. [title] Call out for help if a dog nips at your wrist. [step] If you have the option of fighting your dog, acting calmly and without aggression will help.\nD. This way, the attacker might not be able to strike the dog if it is hurt, and the dog will be off your back. [substeps] If possible, run behind a building to minimize the possibility of injury.","prompt":"[header] How to fight off a dog [title] Stay on your feet and shout for help. [step] If you let the dog jump on you or bring you to the ground, you'll be in a much weaker position and more likely to be seriously hurt. If you're in a city or residential neighborhood, call out for the dog's owner.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5825,"error":null,"output":"A"}
{"question_id":"hellaswag:00005274","item_index":317,"attempt":0,"prompt_hash":"484e13195a28c90433345eab3761e40fe3042c702e763b1f0326fbc82ab31cf8","question":"[header] How to cook rice in pressure cooker [title] Measure out rice and water. [step] Pour your desired measurement of rice into your pressure cooker. Add water according to the type and quantity of rice you are using.\n\nA. [substeps] For example, you should add 15¾ cups (approximately 1.5 l) of water per cup of rice for a 7-8 pound (9.4-1.44 l) pressure cooker. If using a pressure cooker alone, add a pinch of salt to the rice before cooking.\nB. One cup of white rice, for instance, needs 1.5 cups of water. [substeps] Substitute all or part of the water with a liquid for flavoring if you wish, such as chicken broth, vegetable broth, or prepared bouillion.\nC. If you know the ingredients, use that to evaluate the amount of rice that the pressure cooker will have. [substeps] To be more precise, use 1 tbsp.\nD. [substeps] Add water according to the size of your rice (larger may mean larger grains). [title] Fill the pressure cooker with enough water to cover the rice.","prompt":"[header] How to cook rice in pressure cooker [title] Measure out rice and water. [step] Pour your desired measurement of rice into your pressure cooker. Add water according to the type and quantity of rice you are using.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5824,"error":null,"output":"B"}
{"question_id":"hellaswag:00001945","item_index":318,"attempt":0,"prompt_hash":"25cff18eaba58f89f41c057bf11763aed4c92743ba956cbfd8c3c1673a6a2101","question":"We see a lady sitting holding a cat and talking. The lady shows us nail clippers. the lady\n\nA. then takes her contacts out and puts them back in.\nB. unrolls the cat and puts it on her lap.\nC. cleans cat nails with a white tool.\nD. clips her cats claws.","prompt":"We see a lady sitting holding a cat and talking. The lady shows us nail clippers. the lady","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5824,"error":null,"output":"A"}
{"question_id":"hellaswag:00010024","item_index":319,"attempt":0,"prompt_hash":"f77cbc9dd3df514cf2721904a146825901ab4ebf578231261b3440e66cb0678b","question":"[header] How to treat chin acne [title] Examine your lifestyle. [step] A variety of factors can contribute to the development of acne around the chin. Examine your own lifestyle to look for the culprit of your breakouts.\n\nA. [substeps] Chronic feelings of inadequacy often manifest itself as excessive pressure in your temple, which can intensify hairline breakouts. If you continue to feel inadequacy, you may have a problem with the fact that your body has a blocked tear duct.\nB. [substeps] Are you excessively clean? This could include showering less often, going out in public, or and quitting smoking. Are you sober often? These things often affect the development of acne on the nose and mouth.\nC. Call your doctor if you discover : [substeps] You drink four cups a day, or more if you do not. You are obese, carrying a heavy load, or drive heavy trailers.\nD. If certain habits can be identified as the cause, they can altered or eliminated to lessen the severity of breakouts. [substeps] Are you a heavy make-up user? Greasy or heavily applied make-up can clog pores and have been known to cause breakouts.","prompt":"[header] How to treat chin acne [title] Examine your lifestyle. [step] A variety of factors can contribute to the development of acne around the chin. Examine your own lifestyle to look for the culprit of your breakouts.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5825,"error":null,"output":"D"}
{"question_id":"hellaswag:00008163","item_index":320,"attempt":0,"prompt_hash":"f9b0e1dd97a21a8cc51372e5ad4c27e8aa937fb787919bdb646e268a4456f8d2","question":"[header] How to be positive when surrounded by negativity [title] Avoid taking things personally. [step] People who speak negatively oftentimes have a bad habit of doing so. It may be aimed directly at you, but more likely it is merely about the myriad of other factors which are going on in that person's life.\n\nA. It is difficult to face an onslaught of negativity, and especially so when you are the target of it. But keep in mind that the complaints are likely just a way for the individual to vent.\nB. [substeps] You may also be coming across as bad if you shift the focus across from yourself. This may include making remarks about others having things others don't like, sharing opinions that belong to another person, or expressing opinions that you disagree with.\nC. Reacting negatively towards the person you care about has little to do with you representing themselves to this person, or acquiring their respect. [substeps] If you, for example, remember when your mom always told you that you didn't look smart to begin with, consider confronting her on the issue again.\nD. Listen to your gut and often call on help to truly step outside of your comfort zone. [substeps] Your own emotional investment is not a factor in your life and should be taken seriously as it would be impossible to make someone change.","prompt":"[header] How to be positive when surrounded by negativity [title] Avoid taking things personally. [step] People who speak negatively oftentimes have a bad habit of doing so. It may be aimed directly at you, but more likely it is merely about the myriad of other factors which are going on in that person's life.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5848,"error":null,"output":"A"}
{"question_id":"hellaswag:00005851","item_index":321,"attempt":0,"prompt_hash":"e553f550ff0faa9cbe0b71eda6589fd04bec5f251058baf1e8df164ecbfdaedd","question":"[header] How to cross the street [title] Cross only at designated crosswalks. [step] Look for painted lines on the roadway that indicate a safe crosswalk that allows pedestrians to cross the street. In the united states, these will always be painted in white and perpendicular to the street.\n\nA. [substeps] If you are at a stop sign, there will usually be a waiting period after you cross. Check your local time and traffic meter as well.\nB. Watch out for signs from expandable businesses or crime within the area that transmit a message of intent. If someone on whom you are attempting to cross is relatively safe to cross, you can check whether the street has been properly painted by a professional.\nC. [title] Stand squarely in front of a wooden sign attached to a pole for a crosswalk. [step] This sign should say \" crosswalk.\nD. If you are in another country, you may have a different system in place; check with your local transportation department for more information. [substeps] Some crosswalks use several rectangular \" blocks \" of paint to indicate where you should walk across, while others use two parallel lines to designate the crossing area.","prompt":"[header] How to cross the street [title] Cross only at designated crosswalks. [step] Look for painted lines on the roadway that indicate a safe crosswalk that allows pedestrians to cross the street. In the united states, these will always be painted in white and perpendicular to the street.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5847,"error":null,"output":"D"}
{"question_id":"hellaswag:00008768","item_index":322,"attempt":0,"prompt_hash":"e16a8db71d09ded26625983c264b6d28d8eacb16e426c7f44431ab09cce9b659","question":"[header] How to walk safely at night [title] Keep your head up. [step] When walking alone at night, make sure to keep your head up and look forward. This will help you be alert and aware of your surroundings.\n\nA. If you are in a particularly dark area, you won't be able to see through the dark. [substeps] If you have a pair of goggles and such, you can always turn on a flashlight to make sure you can see through the lenses at night.\nB. Do not look down or away, and only make brief eye contact with others. [substeps] To avoid looking distracted, do not walk with your cell phone in your hand.\nC. Studies show that walking alone sometimes causes drowsiness. [substeps] Don't look out for fallen leaves, twigs, or other places to hide.\nD. Consider wearing headbands to keep your head up and even more alert during your walk. Make sure those band holders are securely in place so that you don't lose them.","prompt":"[header] How to walk safely at night [title] Keep your head up. [step] When walking alone at night, make sure to keep your head up and look forward. This will help you be alert and aware of your surroundings.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5855,"error":null,"output":"B"}
{"question_id":"hellaswag:00005073","item_index":323,"attempt":0,"prompt_hash":"991d776a23882a592c20c9d3b98e3d5a2b70a40c013a03f66654359d0b381d77","question":"[header] How to cancel a real estate contract [title] Recognize the difference between buyers and sellers. [step] If you are in the market to buy a house, it should be easy to break off an arrangement with a real estate agent. As a seller, however, you will find that your agent has invested considerable money toward advertising your property and might be disinclined to let you go.\n\nA. To maintain a good record of customer service, most will allow you to end the relationship, but you could encounter some difficulties. [title] Look for an opt-out clause.\nB. If this is the case, you will have to agree to the terms of the contract as well as pay out of pocket. [title] Obtain one loan from a seller who can offer you good credit.\nC. [substeps] By selling your home, you bring in that money. Although it is not part of your total purchase price, it certainly gives you and your agent considerable leverage in selling your home.\nD. You must identify and address the person you wish to cancel your contract with a reasonable degree of respect and professional demeanor. [substeps] They may be able to provide you additional information or suggest some alternatives to make the matter easier.","prompt":"[header] How to cancel a real estate contract [title] Recognize the difference between buyers and sellers. [step] If you are in the market to buy a house, it should be easy to break off an arrangement with a real estate agent. As a seller, however, you will find that your agent has invested considerable money toward advertising your property and might be disinclined to let you go.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5850,"error":null,"output":"A"}
{"question_id":"hellaswag:00009848","item_index":324,"attempt":0,"prompt_hash":"b2396a2c7a3e08acff7da347df4e936464324ecb687984ba796d623a0b499854","question":"[header] How to form lasting friendships [title] Find a new friend. [step] Knowing where to look for like minded people and new friends can be a challenge. Despite this challenge, there are many places you can go to make new friends.\n\nA. Both meet new people and befriend them. [substeps] If you can find other people on a list of people who have similar interests, you can join them.\nB. [substeps] The kind of friendship that seems cool and friendly is usually lonely and is meant to mesh with an existing friendship. It may take a little work, which is okay.\nC. Keep some of these tips in mind when considering where you might be most likely to meet interesting new people and make lasting friendships : [substeps] You could join a group that is centered around one of your interests. For example, if you are into table top gaming, you might head down to your local game store and start playing with the people there.\nD. Most of these places can have the same people and processes. It's important to start with friends who are busy or with their own lives already filled with whatever interests them.","prompt":"[header] How to form lasting friendships [title] Find a new friend. [step] Knowing where to look for like minded people and new friends can be a challenge. Despite this challenge, there are many places you can go to make new friends.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5846,"error":null,"output":"C"}
{"question_id":"hellaswag:00004673","item_index":325,"attempt":0,"prompt_hash":"a099f4e9d65ec91397f08bcc4a8893c3859b64c77aae62e30800810d4dfd1761","question":"[header] How to choose shampoo for scalp psoriasis [title] Know if you need coal tar shampoo. [step] Coal tar is the oldest treatment for scalp psoriasis. However, when it is used alone, it's only as effective as a mild to mid-potency topical corticosteroid.\n\nA. They are effective as additions to combination therapy, though. It can help slow the growth of the flaky skin cells on your scalp and help restore your skin's health.\nB. You may need to consult a doctor or dermatologist, particularly if you have skin psoriasis. [title] Look for a shampoo with a non-comedogenic texture.\nC. [title] Look for a shampoo that concentrates on 80% beta-carotene. [step] This percentage should be clearly represented in the label at the bottom of the shampoo bottle.\nD. [substeps] Coal tar is located in the outer layer of the scalp and hair around the wrinkles. At home, a scraping tool such as a dull razor blade and toothbrush will suffice.","prompt":"[header] How to choose shampoo for scalp psoriasis [title] Know if you need coal tar shampoo. [step] Coal tar is the oldest treatment for scalp psoriasis. However, when it is used alone, it's only as effective as a mild to mid-potency topical corticosteroid.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5847,"error":null,"output":"B"}
{"question_id":"hellaswag:00003111","item_index":326,"attempt":0,"prompt_hash":"e3f2c9048c08259219266af485312a96d47c6b895b77879aafd03c4ac4e11959","question":"The camera pans into a man and leads into shots of him running as well as pictures of him. A camera then shows a large group of people running and the camera panning around the man. more shots\n\nA. are shown of him performing various tricks as well as speaking to the camera and stopping to play with a dog.\nB. are shown of people running and waving to the camera.\nC. are shown of the people playing soccer and running around the area and ends with text across the screen.\nD. are shown as well as clips from a race and people racing on a dirt bike.","prompt":"The camera pans into a man and leads into shots of him running as well as pictures of him. A camera then shows a large group of people running and the camera panning around the man. more shots","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5852,"error":null,"output":"B"}
{"question_id":"hellaswag:00003791","item_index":327,"attempt":0,"prompt_hash":"7d76fa8d3ac2a3f4f7194f31ea7eac6d1cdd04680683cdc1276cb0364f0ab265","question":"[header] How to read cat food labels [title] Look for net weight. [step] In the u.s., many brands abbreviate this as \" net wt \" and list it at the bottom of the front of the bag or can. This is the weight of the contents of the package.\n\nA. [substeps] Net is used to indicate a container weight. Some companies, such as amazon, sell these under different names.\nB. For example, a net weight of 3 oz. ( 85 g) means the package contains three ounces/85 grams of food.\nC. Net weight can be anywhere from 0.57.5-1.119 kg. For instance, if you check a 12 ounce package of litter or pee tube from a local pet store, it should go in the pedometer (m-h) range.\nD. [substeps] Net weight varies by brand, so you'll probably need to go to a local supermarket, department store, or even pharmacy for more options. American express may be accurate with net weight, although not all stores are.","prompt":"[header] How to read cat food labels [title] Look for net weight. [step] In the u.s., many brands abbreviate this as \" net wt \" and list it at the bottom of the front of the bag or can. This is the weight of the contents of the package.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5847,"error":null,"output":"B"}
{"question_id":"hellaswag:00006273","item_index":328,"attempt":0,"prompt_hash":"20032e499fd106d6183b838787ee8ebdaf1883690d0d3958e7b0260e529cc524","question":"[header] How to edit photos using google drive [title] Choose a picture. [step] Take a picture (on a phone) and send it to yourself via gmail, so that you can open it up on your computer. [title] Open google drive.\n\nA. [step] If your account is already logged in, follow the prompts to get your google drive package ready for processing (ie. Login with any google account you have).\nB. [substeps] Go to the left of the screen and see a red button that says \" create \". Under all of the options, find a red colored document folder named \" google drawings.\nC. [step] This is where you will save your photos if you send them. [title] Navigate to the location where you want to edit the photos.\nD. [step] You can open google drive on a windows or mac computer. [title] Go to the \" photos \" tab.","prompt":"[header] How to edit photos using google drive [title] Choose a picture. [step] Take a picture (on a phone) and send it to yourself via gmail, so that you can open it up on your computer. [title] Open google drive.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5848,"error":null,"output":"C"}
{"question_id":"hellaswag:00000843","item_index":329,"attempt":0,"prompt_hash":"0872badc6ef1e6adf3282605088bb632a03a46e6590114e65600c00d9175abfb","question":"A person in a furniture store is talking about popular furniture made from reclaimed wood. He shows rustic, contemporary and traditional type of furniture that is in his store. he\n\nA. demonstrates some of the kinds of flooring used in the furniture makeover.\nB. discusses to other persons about the rafting experience.\nC. continues talking in his store and shows the furniture recliner and parrot.\nD. shows the high quality reclaimed wood and the different furniture made from it.","prompt":"A person in a furniture store is talking about popular furniture made from reclaimed wood. He shows rustic, contemporary and traditional type of furniture that is in his store. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5853,"error":null,"output":"D"}
{"question_id":"hellaswag:00009977","item_index":330,"attempt":0,"prompt_hash":"3a807a413eb55c65e85085b83b9575d0ffd871e3755eb197f484a460ea03d876","question":"[header] How to dress business casual for the plus size woman [title] Define business casual. [step] The very first question for many people is, \" what is business casual? \" it seems like a contradiction and definitions can vary greatly from one person to another. One thing is sure, business casual is not suits and jackets.\n\nA. On the other hand, it's not jeans and flip flops, but somewhere in between. [substeps] The best way to define your specific company's interpretation of business casual is to look around at your co-workers.\nB. Business casual is the single term for your clothing that describes everything from your cut to how you dress. [substeps] Ask yourself what is business casual based on have: business casual? Practice wearing a suit or a dress, and adjusting if you prefer a more conservative look.\nC. When you're trying to create business casual, remember, this is what you're doing. You're creating business casual to reflect who you are.\nD. However, you wouldn't go for khakis because they are made from spandex, but you wouldn't have to wear t-shirts either. Here are some steps for you to take to get into business casual : [substeps] Look for short business pants that fit comfortably (but quickly).","prompt":"[header] How to dress business casual for the plus size woman [title] Define business casual. [step] The very first question for many people is, \" what is business casual? \" it seems like a contradiction and definitions can vary greatly from one person to another. One thing is sure, business casual is not suits and jackets.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5850,"error":null,"output":"A"}
{"question_id":"hellaswag:00004742","item_index":331,"attempt":0,"prompt_hash":"cc6866a70d3af7a997278e8743c28bd597d3277b1539d369873ba01564bed97e","question":"[header] How to wash a human hair weave [title] Saturate your weave with warm water. [step] Run warm water over your entire weave in a downward motion near where your tracks are until the weave is completely wet. You can do this by holding your head underneath the sink faucet but it's probably easier just to get in the shower.\n\nA. Make sure the immersion water is warm enough to completely saturate your weave. [substeps] The ideal immersion water temperature should be no less than 70 ° f (27 ° c).\nB. [substeps] Place your head under the stream of lukewarm water and wring your hair out as much as possible while still holding your head over the sink. [title] Add some conditioner to your weave to prevent damage.\nC. [title] Apply shampoo and rinse it out. [step] Squirt a small amount of a hydrating, clarifying, sulfate-free shampoo into your hand and massage it gently into the length of the weave.\nD. This will keep the weave saturated in the water and is harder to control when rinsing it. [substeps] Use a gentle shampoo to wash the weave, once you've used the hot water water.","prompt":"[header] How to wash a human hair weave [title] Saturate your weave with warm water. [step] Run warm water over your entire weave in a downward motion near where your tracks are until the weave is completely wet. You can do this by holding your head underneath the sink faucet but it's probably easier just to get in the shower.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5848,"error":null,"output":"C"}
{"question_id":"hellaswag:00009290","item_index":332,"attempt":0,"prompt_hash":"8e1daf4a23467fa8ff150d1abe690db1dcfcbfc6a5b1e482daf9ae5b5ec34174","question":"[header] How to install a sprayer to your toilet to clean cloth diapers [title] Gather all materials, and make sure that you've got everything. [title] Lay everything out as it will be installed : [step] Stop tee, shut-off valve, supply line, hex nipple, coupling, sprayer. [title] Wrap all of the exposed threads with pipe thread tape.\n\nA. [title] Connect the coupling to the hex nipple and tighten. [title] Connect the supply line to the open side of the hex nipple.\nB. [step] When finished wrap loose threads or pipe points around the pipe threads to tighten connections. [substeps] Load pipe joints and hoses into pipe joints.\nC. [step] Or snip a very small hole in one end of the pipe thread tape so that it will be able to open easily. [title] Attach a small hex nipple cable to the toilet pipe.\nD. [step] Try to keep the pipe threading and joint cap separated and tightly sealed. [title] Lay down the loading hose : [step] Fold the end under the cap.","prompt":"[header] How to install a sprayer to your toilet to clean cloth diapers [title] Gather all materials, and make sure that you've got everything. [title] Lay everything out as it will be installed : [step] Stop tee, shut-off valve, supply line, hex nipple, coupling, sprayer. [title] Wrap all of the exposed threads with pipe thread tape.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5849,"error":null,"output":"A"}
{"question_id":"hellaswag:00006316","item_index":333,"attempt":0,"prompt_hash":"ed0a7f19cda7793c738bce2358985e04d9d7247a9cfea4742b2217016cd21146","question":"[header] How to succeed in a pageant [title] Enter the pageant and make the deadlines. [step] Be sure to write everything neatly and be honest. Judges use applications to create questions for the interview (if there is one) and on-stage questions.\n\nA. [substeps] Be sure that people are prepared for the pageant and on-stage interviews. Set a schedule for the pageant and perform any necessary steps.\nB. Be sure to ask the questions while your cover letter is still fresh in the minds of the pageant entrants. [title] If you do not get the job, apply.\nC. [substeps] Make a habit of having ready-made questions in your application packet. Bring that aside and review the show's schedule.\nD. It's much better to seem boring than to seem like a liar! [title] Find a good dress to flatter your body. [step] It doesn't need to be super expensive or fancy.","prompt":"[header] How to succeed in a pageant [title] Enter the pageant and make the deadlines. [step] Be sure to write everything neatly and be honest. Judges use applications to create questions for the interview (if there is one) and on-stage questions.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5849,"error":null,"output":"D"}
{"question_id":"hellaswag:00007274","item_index":334,"attempt":0,"prompt_hash":"f9362a6cd89d9cc83b74ee93c03656a23d9ccba5b4119d2b9311f97b157ff824","question":"[header] How to make fake rocks for your pond [title] Bend the chicken wire fencing into the shape that you need. [step] Cut, fold, bend, do whatever you have to do to get it into the shape you want. Be sure to wear leather gloves for this! [title] Set the wire frame down on a piece of plastic.\n\nA. [step] I called it a fish frame because the plastic can easily be removed. This means you can just glue the wire frame to the frame, then remove it.\nB. [step] Use a serrated knife to cut a large slit of plastic 6 cm across width and 12 cm deep about six inches away from the edge of the plastic frame. [title] Thread 2 small pieces of small plastic mesh on 1 of the strips of plastic mesh.\nC. [step] You'll be using the cans of foam insulation for the next step (possibly up to 3 cans). [title] Spray the cans of insulation onto the wire frame.\nD. [step] Put some rocks on either side of the frame, and glue these rocks together using pvc joint compound. [title] Allow the pvc joint compound to cure.","prompt":"[header] How to make fake rocks for your pond [title] Bend the chicken wire fencing into the shape that you need. [step] Cut, fold, bend, do whatever you have to do to get it into the shape you want. Be sure to wear leather gloves for this! [title] Set the wire frame down on a piece of plastic.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5849,"error":null,"output":"D"}
{"question_id":"hellaswag:00009154","item_index":335,"attempt":0,"prompt_hash":"21dab90a64e8f1e2a404e1d4ae4c7198075fb081bfa914963d099c4a2fdde3b8","question":"[header] How to get rid of wasps in your house [title] Gather your materials. [step] This is easy to make and can be put in the corner of the room that has a wasp problem. [substeps] Scissors empty plastic bottle (2l) tape bait (watered down dish soap ) [title] Remove the bottle cap and cut through the bottle a third of the way down.\n\nA. [step] Be sure to do this carefully with your scissors. You can also use a razor blade if the scissors don't seem to be cutting through the bottle.\nB. [step] This lets the wasps have the chance to escape. Be very careful here--wasps like to drown in water for awhile, so don't have your black bottle either.\nC. [title] Pour boiling water into the bottle through the piercing holes. [step] Leave just a little room, within there may be a nest.\nD. [step] Your box may have a cover for it; simply remove the lid and cut through that half. [substeps] It is important to go slowly as you are cutting.","prompt":"[header] How to get rid of wasps in your house [title] Gather your materials. [step] This is easy to make and can be put in the corner of the room that has a wasp problem. [substeps] Scissors empty plastic bottle (2l) tape bait (watered down dish soap ) [title] Remove the bottle cap and cut through the bottle a third of the way down.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5848,"error":null,"output":"A"}
{"question_id":"hellaswag:00007736","item_index":336,"attempt":0,"prompt_hash":"3a3e08a4ae871717aa60ef04bcf54fe52acf5d20a89167124836921893ed53d2","question":"[header] How to recover from an emotional affair [title] Forgive yourself. [step] While forgiving others can be challenging, it's often even harder to forgive yourself. Remember that if you had known the path of your actions and their consequences, you probably would not have done what you did.\n\nA. Take some time to live in the past and let go of those emotions. For example, if you had experienced a miscarriage, forgiveness would be easy.\nB. Forgiveness in someone else can only serve to make it harder. [substeps] Cheating can be very emotional, and can even be worse.\nC. To begin forgiving yourself, admit that you messed up or made a mistake. Making mistakes is part of being human and no one is exempt from it.\nD. In the moment, forgive yourself for all the things you could have done differently. [substeps] Though you may have felt wronged, forgiving yourself for your actions will take time and effort to live.","prompt":"[header] How to recover from an emotional affair [title] Forgive yourself. [step] While forgiving others can be challenging, it's often even harder to forgive yourself. Remember that if you had known the path of your actions and their consequences, you probably would not have done what you did.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7109,"error":null,"output":"C"}
{"question_id":"hellaswag:00006102","item_index":337,"attempt":0,"prompt_hash":"a0307d81663ea4305999867b48c8f1147e615c9d6c99578f3ef6846ca3a4dace","question":"[header] How to choose an unconventional engagement ring [title] Determine the colors they prefer. [step] Engagement stones come in a variety of different colors. Ask them, or take notice of their favorite colors and choose a ring that reflects it.\n\nA. [substeps] The color wheel goes round and round, but the chosen color palette changes when you try new jewelry. Some colors that are unfamiliar to you may be too subdued for a traditional engagement ring.\nB. For instance, a ring that is white or blue will always be white whereas a ring with a clever look or a unique theme will always be brown (gold) or red. [title] Try an unconventional ring : [step] A classic engagement ring will feature all the same colors.\nC. Take notice of the current color of their jewelry and base your decision on that. Conventional rings are usually transparent, so consider getting a different color for your ring.\nD. Accessorize with ties, buttons, necklaces, etc. [substeps] Look for a vintage or classic look when choosing an unconventional engagement ring.","prompt":"[header] How to choose an unconventional engagement ring [title] Determine the colors they prefer. [step] Engagement stones come in a variety of different colors. Ask them, or take notice of their favorite colors and choose a ring that reflects it.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7110,"error":null,"output":"C"}
{"question_id":"hellaswag:00003993","item_index":338,"attempt":0,"prompt_hash":"4adc7aff9b3ded7b43a639194d8def253a3376c1e3baf1cc9d7fdf7a82f804e0","question":"[header] How to play castanets [title] Choose the right size castanets. [step] This is largely a matter of preference, as the string connecting the shells of your castanets are adjustable. The size of your castanets will also influence the tone you create.\n\nA. [substeps] Castanets that are too small will add to the melody. Larger sizes will place too much emphasis on tempo.\nB. [substeps] Stocks of castanets tend to be larger (often if not more), while stocks of castanets tend to be smaller (often even costing big bucks). Buying castanets too small can affect how your bass sound plays out.\nC. [substeps] After inserting the shells of your castanets into the shells you will now need to find a 3/4 \" wood or wooden base for the shells. Choose a base like the sand under which the shells break or clump.\nD. Generally, you'll want a pair that fit comfortably in your hand and can be hidden easily from view in your palm. [substeps] Sizes range from three, which is usually for children or to achieve high pitches, up to nine for larger hands and deeper tones.","prompt":"[header] How to play castanets [title] Choose the right size castanets. [step] This is largely a matter of preference, as the string connecting the shells of your castanets are adjustable. The size of your castanets will also influence the tone you create.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7111,"error":null,"output":"D"}
{"question_id":"hellaswag:00001257","item_index":339,"attempt":0,"prompt_hash":"27abd78989eb27a4be5dd8ff4d42ceacac03e1be7961ac9749c659738a7bc972","question":"A woman in a red shirt is sitting down talking. A picture of an older man is shown on the screen. a man and a woman\n\nA. are sitting at the table talking.\nB. are playing volleyball outside in the sand.\nC. start hitting ping pong balls around a table.\nD. play beer pong standing in a room fighting.","prompt":"A woman in a red shirt is sitting down talking. A picture of an older man is shown on the screen. a man and a woman","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7111,"error":null,"output":"A"}
{"question_id":"hellaswag:00004159","item_index":340,"attempt":0,"prompt_hash":"fe92ff24d06e8e90be460d778f23eb5c5597225f9d614f36bd71bd48f1f23ea3","question":"[header] How to write a speech introducing yourself [title] Make an outline of your speech. [step] Start by making a skeletal draft of your main points. Strip the speech down to its bare bones to determine what is most important to say, and in what order you should deliver those facts.\n\nA. This is the basic structure which you will build your speech around. [substeps] State your name in the very first sentence of your speech.\nB. [substeps] Keep it short, as every paragraph should be at least 50 minutes long. You can take breaks to reflect on your speech, and add these to the outline.\nC. [substeps] It's bad form to instead write in your own words. This creates a skeptical feel to the speech, and will only build on your ideas.\nD. For instance, write, \" speak for me. \" [substeps] If you choose to save the outline by writing out your introduction yourself, you can leave some detail to ease your audience into what your main points are.","prompt":"[header] How to write a speech introducing yourself [title] Make an outline of your speech. [step] Start by making a skeletal draft of your main points. Strip the speech down to its bare bones to determine what is most important to say, and in what order you should deliver those facts.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7109,"error":null,"output":"A"}
{"question_id":"hellaswag:00006333","item_index":341,"attempt":0,"prompt_hash":"fef6d2bb58f89d78ba29849560e1d8fa90be00d80ee74b7b7cd56799a8ad7e7d","question":"[header] How to take care of a labrador puppy [title] Know, before hand, that you have the resources, and time to take care of a lab puppy. [step] If you don't, both you and your puppy could suffer. [title] Choose the lead, collar, bowl, food type, and sleeping places.\n\nA. [step] Also make sure your brush and pen is clean. Also make sure your model puppy brushes in dirt.\nB. [step] Make sure you have quality cleansers and that the puppy is not easily frightened of anything. [title] Pick a sleeping area to give your puppy access to the toilet.\nC. [step] Provide the following things for your puppy : [substeps] During puppies sleep, they need solid human food. During the day, go for canned food.\nD. [step] This will be where the puppy will go for the rest of his/her life, so choose wisely. Remember, that food and water needs to be kept under shelter.","prompt":"[header] How to take care of a labrador puppy [title] Know, before hand, that you have the resources, and time to take care of a lab puppy. [step] If you don't, both you and your puppy could suffer. [title] Choose the lead, collar, bowl, food type, and sleeping places.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7109,"error":null,"output":"D"}
{"question_id":"hellaswag:00003125","item_index":342,"attempt":0,"prompt_hash":"2286930407812a598a5c46dfa692efe9a8683e9402ab9be9623fc6eddeb19a85","question":"A crew of men clean a red car with clothes. two women\n\nA. wash two tires with a cleaning solution in a sink.\nB. wash a white car using a hose.\nC. are on the ground in an smoking area playing table tennis.\nD. talks inside a car washing shop where there are people waiting.","prompt":"A crew of men clean a red car with clothes. two women","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7109,"error":null,"output":"B"}
{"question_id":"hellaswag:00007194","item_index":343,"attempt":0,"prompt_hash":"900e8121304b94cbf397f232efd07dd3a73e2b47ffd778041c5f4b5d4eff5a21","question":"[header] How to prepare your home for the rainy season [title] Get your roof ready for the rains. [step] Inspect your roof twice per year to avoid costly problems that can escalate into tremendous cost. [substeps] Look for cracks along the ridge of your roof and where your shingles fold over to form the cap.\n\nA. It's vital for your roof structure to be constructed appropriately for the upcoming rains. This can be accomplished with basic care, but should only be done once or twice a year, especially if there's a severe increase in rainfall.\nB. Inspect the valleys of your roof (the area of your roof with a downward slope). Make sure that the sheet metal flashing does not have any holes or rusty spots.\nC. Make sure the roof is acrylic, known for its pointed corners and rough edges. [title] Make sure the water can drain from the eaves of your house.\nD. Use the energy and materials that you have now taken in to remove plastic or stone caps that reinforce the roof (which can be put into reuse) or any other dramatic build-up. [title] Buy/rent or rent your house for the rainy season.","prompt":"[header] How to prepare your home for the rainy season [title] Get your roof ready for the rains. [step] Inspect your roof twice per year to avoid costly problems that can escalate into tremendous cost. [substeps] Look for cracks along the ridge of your roof and where your shingles fold over to form the cap.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7109,"error":null,"output":"B"}
{"question_id":"hellaswag:00007236","item_index":344,"attempt":0,"prompt_hash":"13eef33f8de019b7b96864a3c3aaf3086a821a3450e8116881b48f0f628025db","question":"[header] How to draw fashion sketches [title] Gather materials. [step] Choose a hard lead pencil (h pencils are best) that makes light, sketchy marks that are easy to erase. Marks made with these pencils also don't indent the paper, which is helpful when you want to add color to the image.\n\nA. There are many different types of pencils-all good for page sketching. [substeps] Pencils work best in glossy and flat drawing.\nB. [title] Sketch the body outline which is often the trickiest part of drawing a fashion sketch. [title] Draw the figure outline by drawing lines.\nC. [title] Trace on your drawing with an eraser if needed. [step] Even a little smudge is needed.\nD. A good quality eraser and thick paper are also important materials to have if you want your sketch to look professional. [substeps] If you don't have the right type of pencil, you can do a sketch with a number 2.","prompt":"[header] How to draw fashion sketches [title] Gather materials. [step] Choose a hard lead pencil (h pencils are best) that makes light, sketchy marks that are easy to erase. Marks made with these pencils also don't indent the paper, which is helpful when you want to add color to the image.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7109,"error":null,"output":"D"}
{"question_id":"hellaswag:00007545","item_index":345,"attempt":0,"prompt_hash":"b6c950d6eb43a78a69c74a877e906e31cf86f4ea6963a2b08c7efec381591b3b","question":"[header] How to exfoliate, steam and use face masks [title] Cleanse your face. [step] It's important to start with a clean face before you exfoliate. Wet your face with warm water, and massage a cleanser that fits your skin type into your skin.\n\nA. Rinse the cleanser off with lukewarm running water. While soap helps to remove dirt, it won't scrub off with hot water, which will make your mask resistant to removal.\nB. Rinse your face off with cold water when you're done. [substeps] Your face is more sensitive to chemicals than it is your skin, which can cause clogged pores and facial blemishes.\nC. This will help break up the tonsil, the dry, flaky outer layer of skin and free up your pores and prevent pimples from forming. [substeps] If you have oily skin, you may want to wash your face more often to get rid of the sebum.\nD. Rinse off the cleanser with lukewarm water. [substeps] You can pat your face with a clean towel so it's not dripping but it's not necessary to dry it completely.","prompt":"[header] How to exfoliate, steam and use face masks [title] Cleanse your face. [step] It's important to start with a clean face before you exfoliate. Wet your face with warm water, and massage a cleanser that fits your skin type into your skin.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7110,"error":null,"output":"D"}
{"question_id":"hellaswag:00002288","item_index":346,"attempt":0,"prompt_hash":"57cd2e6d47aa00fc36c6d346eda8321e6857be2216091f755b1df9263e4a5c71","question":"The video begins with a title sequence. a young man\n\nA. prepares to black out.\nB. is standing outside with a basketball in his hand, alternating between shots of dribbling for the ball.\nC. is shown in a gym performing tricks with a jump rope as music plays in the background.\nD. is seen talking continuously about slamming the mouth of a chimpanzee into the camera.","prompt":"The video begins with a title sequence. a young man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7108,"error":null,"output":"C"}
{"question_id":"hellaswag:00004676","item_index":347,"attempt":0,"prompt_hash":"6f3bafd341cc1c77e4a741d1992597771e14bf2b424bedc673ec854cf58fa184","question":"[header] How to apply false eyelashes [title] Size the lashes. [step] Before gluing the lashes on, you will need to make sure they are not too wide for your eye. Hold the lash strips up against your eyelid, and trim them down on the sides if necessary.\n\nA. [substeps] If the lashes are too long for your personal taste, consider trimming the individual lashes down to achieve a more natural look. Lashes should be longer toward the outer corner of the eye.\nB. [substeps] Keep in mind it might be quite a daunting task to pull the lashes all together in one big pair. If this is the case, measure the length and width of your lashes before you begin.\nC. You may also have a makeup consultant come in and measure the lashes directly to determine if they are too small. [substeps] If you want to take care of lashes that are too small for your eyes, then do not trim them in this step.\nD. [substeps] Eyelashes are meant to cover one part of the eye, but for the desired effect, you must be able to see both eyelashes at once. To achieve this, ensure that you have the correct handle.","prompt":"[header] How to apply false eyelashes [title] Size the lashes. [step] Before gluing the lashes on, you will need to make sure they are not too wide for your eye. Hold the lash strips up against your eyelid, and trim them down on the sides if necessary.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6707,"error":null,"output":"A"}
{"question_id":"hellaswag:00000655","item_index":348,"attempt":0,"prompt_hash":"633bfdfbc71abd702ea39d476bffd585ec4433daac256a52474d398c8785a8ee","question":"A man performs a long jump on a track and into a sandpit in front of judges and onlookers on the sidelines. after the man\n\nA. is shown he gets up from his seat and shakes hands with his name and friends, walks off stage where people applaud and wave him on stage and live photos appear.\nB. on the track becomes a golden retriever.\nC. falls down with two others cheering as he continues to jump on the track.\nD. lands the jump he explodes in celebration by clenching his fists, after which a slow replay of the jump plays and a scoreboard displays his score.","prompt":"A man performs a long jump on a track and into a sandpit in front of judges and onlookers on the sidelines. after the man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6707,"error":null,"output":"D"}
{"question_id":"hellaswag:00009999","item_index":349,"attempt":0,"prompt_hash":"adc4e20ca474e4fd58221cb4b6bac9be7f490dd38cffaa1ce016c537565ce812","question":"[header] How to use tucks pads for hemorrhoids [title] Prepare a bath. [step] Before you use a tucks ® pad, you will need to ensure that the anal area is clean. The best way to clean the anal area is to take a warm sitz bath.\n\nA. Allow yourself enough time in the bath to relax. [substeps] The recommended bath time should be a month for most hemorrhoids, although some states may require more for even larger ones.\nB. This will warm up the area and get the colon and rectum warm to the touch. [substeps] You should also have a thermometer handy when you bathe your hemorrhoids to help determine how long they will last.\nC. Sitz baths only require a few inches of water to soak the anus but you can take a full bath if you prefer. If you prefer the traditional sitz bath, just fill the tub with a few inches of water.\nD. In a sitz bath, place a soft towel around the area and prop the pad in the water with a few drops of lemon or mint oil. Put the pad in the bath as the water goes on, which slowly relaxes it.","prompt":"[header] How to use tucks pads for hemorrhoids [title] Prepare a bath. [step] Before you use a tucks ® pad, you will need to ensure that the anal area is clean. The best way to clean the anal area is to take a warm sitz bath.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6707,"error":null,"output":"C"}
{"question_id":"hellaswag:00009045","item_index":350,"attempt":0,"prompt_hash":"b1c858ee000f5c44131b5aa6c1116b1b5c602aa1d88c5c96c2b9e15c8b84f333","question":"[header] How to make a green tea toner [title] Add 1 green tea bag or 2 tbsp. [step] ( 30 ml) loose green tea leaves into 1 cup (236 ml) of boiling water. [title] Allow the tea to steep for 3 to 5 minutes.\n\nA. [title] Strain the tea and add 1 cup (236 ml) of filtered water to the steep tea. [title] Drain the water and add 1 teaspoon of tea tree essential oil.\nB. [title] Strain the loose green tea by pouring it through a fine strainer. [step] If you feel the liquid bending, you do not need to strain it.\nC. [title] Remove the tea bag and pour the liquid into an air-tight container. [step] If you are using loose green tea leaves, pour the tea through a strainer or coffee filter into an airtight container.\nD. [title] Add 1 cup (237 ml) green tea leaves to boiling water. [title] Let the tea steep for 5 minutes then strain out the leaves.","prompt":"[header] How to make a green tea toner [title] Add 1 green tea bag or 2 tbsp. [step] ( 30 ml) loose green tea leaves into 1 cup (236 ml) of boiling water. [title] Allow the tea to steep for 3 to 5 minutes.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6706,"error":null,"output":"C"}
{"question_id":"hellaswag:00001389","item_index":351,"attempt":0,"prompt_hash":"72e1522c4c6f844be503b8fd4a206e2840c02b4658de56a801535ca43a47234c","question":"One of the singers is playing the violin while the audience cheers and claps for him. Then another singer grabs the mic and talks to the audience. the audience\n\nA. claps for him as he continues to talk to the crowd.\nB. applaud and smile as they watch.\nC. turns even flips around and the singers score.\nD. starts clapping and smiling from behind the crowd.","prompt":"One of the singers is playing the violin while the audience cheers and claps for him. Then another singer grabs the mic and talks to the audience. the audience","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6706,"error":null,"output":"A"}
{"question_id":"hellaswag:00001405","item_index":352,"attempt":0,"prompt_hash":"dbd21be5491e668cc760e32610398de843ded9fe60009484723705d4113fccdf","question":"A man is seen speaking into a microphone with another man on stage. one man\n\nA. plays a set of bongos while the other speaks.\nB. then hits the other one with a wooden mallet.\nC. then leads into playing harmonica on the stage while others watch on the side.\nD. sits in front of the man talking into a microphone while another man sits behind him.","prompt":"A man is seen speaking into a microphone with another man on stage. one man","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5971,"error":null,"output":"D"}
{"question_id":"hellaswag:00002167","item_index":353,"attempt":0,"prompt_hash":"130348e74d5242211e2c96ea64bd74c782a69ad10567356391ff58c6803eb6ef","question":"She continues playing her harmonica. There's a young boy and his mother who smile at the lady as she plays. then they\n\nA. continue playing as they hug their new child before walking away.\nB. continue to play mini harmonica's.\nC. all continue to play individually on the strings of their harmonica.\nD. get off the elevator when they reach their level.","prompt":"She continues playing her harmonica. There's a young boy and his mother who smile at the lady as she plays. then they","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5971,"error":null,"output":"D"}
{"question_id":"hellaswag:00004244","item_index":354,"attempt":0,"prompt_hash":"b1809bc7aa60a19d0343e4b27b646e99d1a340c71eca0155e34145054b8d8d3b","question":"[header] How to make a substitute sanitary pad [title] Find a piece of cotton wool around the size of an average sanitary pad. [step] It doesn't matter if the shape or size is perfect. [title] Wrap your cotton wool with toilet paper.\n\nA. [step] Be sure to use a non slip material. A sterile, absorbent toilet paper provides a good barrier.\nB. [step] Fold it tightly, so the two sides are just touching, without any creases. Make sure no moisture (or water) has escaped.\nC. [step] The cotton can be long or short. Just keep in mind that toilet paper must be long, and not too skinny because so many people use that.\nD. [title] Place your substitute sanitary pad on the usual spot on your underwear. [title] Get a large amount of toilet paper.","prompt":"[header] How to make a substitute sanitary pad [title] Find a piece of cotton wool around the size of an average sanitary pad. [step] It doesn't matter if the shape or size is perfect. [title] Wrap your cotton wool with toilet paper.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5972,"error":null,"output":"D"}
{"question_id":"hellaswag:00003700","item_index":355,"attempt":0,"prompt_hash":"f67dfedad255027a2f1f1a3d200e76be4575c10f2f37152987f272443e59f5e5","question":"[header] How to make your own clothing line [title] Think of a name for your clothing line. [step] Write down as many ideas as you can think of, then choose your favorite. [title] Decide on a target audience.\n\nA. [step] Creating your clothing line should be a simple matter of writing at home. The goal might be to attract viewers to your clothing line so that they can see and go looking for your clothing line.\nB. [step] Have friends or family members who are into fashion, have noticed any trends in clothing in their area, and who is a good role model. Be able to spend time with this specific audience, which will also help give your product line a model that potential customers will want to associate with.\nC. [step] Have knowledge regarding your target audience on a daily basis. You may choose one step at a time, while your concerns about impressing a population are not always measured in convention.\nD. [step] This will influence not only your designs, but the prices of your designs too. Are you going to make your designs from scratch or create a recycled clothing line? [title] Start designing clothes.","prompt":"[header] How to make your own clothing line [title] Think of a name for your clothing line. [step] Write down as many ideas as you can think of, then choose your favorite. [title] Decide on a target audience.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5966,"error":null,"output":"D"}
{"question_id":"hellaswag:00003871","item_index":356,"attempt":0,"prompt_hash":"710be40122689b6d09b981e20ffd0ab7bb6345b61bcef0e3279779dad7b487fd","question":"[header] How to make cute everyday hairstyles [title] Smooth your hair back and secure it with an elastic for a basic ponytail. [step] Ponytails are one of the most versatile hairstyles you can wear, and they're the starting point for a number of other styles. All but the shortest hair lengths will work in a ponytail.\n\nA. [substeps] Try wearing your ponytail near the base of your neck for a casual, romantic look. You can even consider wearing a low ponytail with a middle or side part.\nB. If you like to shake out your hair, try fishnet or spandex instead to add some volume to your tresses, then wrap a side of your hair around your ponytail to give it a little more bulk. [title] Use clips to secure your hair up.\nC. Clip back the sides if necessary for quick and easy flipping at the end of the ponytail. For straight or wavy hair, add a small amount of mousse on your roots and at the back of the ponytail.\nD. [title] Brush out your hair from roots to tip. [step] Style the front part of your hair slowly, making sure to work with a comb only.","prompt":"[header] How to make cute everyday hairstyles [title] Smooth your hair back and secure it with an elastic for a basic ponytail. [step] Ponytails are one of the most versatile hairstyles you can wear, and they're the starting point for a number of other styles. All but the shortest hair lengths will work in a ponytail.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5966,"error":null,"output":"A"}
{"question_id":"hellaswag:00003131","item_index":357,"attempt":0,"prompt_hash":"9b7cc80a31034e834063fb3dc960e680c33ff0604cd3105d7e9d27fceef010f6","question":"A man is seen speaking to the camera and standing in front of a table with various tools laid out in front of him. He then is seen wearing a hat, holding a mug, and using a knife around it. he\n\nA. wipes mud all over the table and moves there tools all around.\nB. turns the knife around and demonstrates how it works, cutting the lemons out with a small cutter and shashing them into an alcoholic bite.\nC. continues the process until it eventually cuts off and he finally cups a cup and pours it into the mug.\nD. then uses the knife along a window and is shown several times sharpening the knife and speaking to the camera.","prompt":"A man is seen speaking to the camera and standing in front of a table with various tools laid out in front of him. He then is seen wearing a hat, holding a mug, and using a knife around it. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5969,"error":null,"output":"D"}
{"question_id":"hellaswag:00007357","item_index":358,"attempt":0,"prompt_hash":"72a92502d7daa65852ef259c81b6d511cfaa4553d7e061a69ba6d735b5cf8a70","question":"[header] How to reduce your fall risk with exercise [title] Talk to your doctor and a physical therapist. [step] When you are deciding to start an exercise program, you should talk to your doctor and/or a physical therapist about your plans. This is particularly important if you have mobility issues or trouble with physical activity, as your doctor can help you determine what is safe for you.\n\nA. No matter what your physical fitness level is, you can be more active and get on a program to strengthen your body. A physical therapist can help you come up with an exercise plan to help reduce your risk of falling.\nB. [substeps] Your physical therapist may also be able to suggest other exercise programs that you could try. If you want to start a new exercise program, discuss the options the therapist has expressed with you.\nC. You may also want to consider home safety measures before starting the program. Taking steps to make sure your home is not subjected to any hazards can be a great way to improve your health and minimize the risk of falling at any point in your exercising regimen.\nD. [substeps] The therapist can help you define your fitness goals and form a plan that will be convenient for you while you exercise. The physical therapist can help you describe your exercises to her and other physical therapists so that they can give you guidance.","prompt":"[header] How to reduce your fall risk with exercise [title] Talk to your doctor and a physical therapist. [step] When you are deciding to start an exercise program, you should talk to your doctor and/or a physical therapist about your plans. This is particularly important if you have mobility issues or trouble with physical activity, as your doctor can help you determine what is safe for you.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5966,"error":null,"output":"A"}
{"question_id":"hellaswag:00007686","item_index":359,"attempt":0,"prompt_hash":"705f8309154e50c364d53f084564bd4333595c93d4aff7455e86d0aabe035686","question":"[header] How to make banana fritters [title] Mash the bananas. [step] Peel both of the bananas and slice them into small chunks. Then place the chunks in a sturdy, medium sized bowl and mash them using a potato masher or with a fork.\n\nA. You can also make them into mashed bananas by crushing the bananas with your fingers. [substeps] You should mash for about 20 seconds depending on how big of a chunks you are having.\nB. If you don't have a potato masher, place the slices in a blender or food processor. [substeps] You can also mash with a large spoon, though it may take slightly longer.\nC. [substeps] You can use any type of mash you like, including mashed bananas, cashews, or pudding chicken posaya. If you prefer, you can also crush the fruits before mixing them in.\nD. [substeps] Make sure to use warm milk as opposed to ice cold. Bananas do not ferment all the way through.","prompt":"[header] How to make banana fritters [title] Mash the bananas. [step] Peel both of the bananas and slice them into small chunks. Then place the chunks in a sturdy, medium sized bowl and mash them using a potato masher or with a fork.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5966,"error":null,"output":"B"}
{"question_id":"hellaswag:00005940","item_index":360,"attempt":0,"prompt_hash":"75cb8f75a9e74a24297f97381bdb182f9e8869263bd949a340b2c9b1cea970c8","question":"[header] How to prevent child molestation [title] Teach your children safety basics first. [step] This means covering not only the way to safely cross the street and watch for oncoming traffic but also role-playing situations that mimic the types of things would-be predators might do or say to get a child's attention. Try to tailor this activity to the age and maturity level of your child.\n\nA. [substeps] Allow him to develop basic safety skills. Encourage your child to do so out of the comfort of his own home.\nB. You can teach them the dangers of falling, that can cost them their life, and how to walk safely on sidewalks. [substeps] Use books and packages to explain safety skills.\nC. Some ideas are : [substeps] Identify the traffic or safety hazards that your child is experiencing within a given intersection. Identify the hazards that your child may be facing.\nD. You can even use dolls or other toys to make this less intimidating for your child. [title] Select an age-appropriate book and teach your child the parts of the body.","prompt":"[header] How to prevent child molestation [title] Teach your children safety basics first. [step] This means covering not only the way to safely cross the street and watch for oncoming traffic but also role-playing situations that mimic the types of things would-be predators might do or say to get a child's attention. Try to tailor this activity to the age and maturity level of your child.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5965,"error":null,"output":"D"}
{"question_id":"hellaswag:00004380","item_index":361,"attempt":0,"prompt_hash":"1d059a4489649a5c28e7ee831f9005bf5e77466758769128a4da53239656890b","question":"[header] How to become a scene queen on facebook [title] Get the hair. [step] The hair is the most important step. Make sure to get lots of layers and if your hair is short get some extensions.\n\nA. You don't have to wear them all the time, just for some of your pictures. Most scene queens on facebook will have varying hair.\nB. [title] Don't care if this is your first time getting it, if you want it then ignore it. [step] You will start figuring out who you are, and this will not make much sense to begin with.\nC. If you have long hair add a couple of layers per 2/3 or side of your head. If you have hair that is shoulder length try to get extensions usually put more layers.\nD. When you go to school, try to only have your hair in layers. If you have your hair in braids, just skip this step.","prompt":"[header] How to become a scene queen on facebook [title] Get the hair. [step] The hair is the most important step. Make sure to get lots of layers and if your hair is short get some extensions.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5968,"error":null,"output":"A"}
{"question_id":"hellaswag:00004955","item_index":362,"attempt":0,"prompt_hash":"cf5041b3f9018c0e48072c11b9f74208e5a28c8f36ce6a79a8bfc5fb5926d7c7","question":"[header] How to buy a mattress [title] Visit mattress websites to see what's offered. [step] If you haven't purchased a mattress in a while, it's good to see what options are available before going to a store. [substeps] Check prices online to see what you feel is reasonable based on what is being offered.\n\nA. Look for online deals, but check with your local grocery and drug stores if you have questions about how much that can cost. You can say something like \" hi, my favorite mattress will cost $800 \".\nB. Mattress brands are often coming out with new styles of mattresses including ones that offer adjustable firmness levels and temperatures. Decide how high-tech you want your mattress to be, as some of these may only be available at specialty stores or online.\nC. There are websites like handylist.com that have more than a handful of suppliers. [title] Get a stylish, functional mattress.\nD. Even if the prices are lower than what you need, you don't want to get stuck buying a mattress that is under warranty. Sites like ebay often offer discounts on mattresses of different prices.","prompt":"[header] How to buy a mattress [title] Visit mattress websites to see what's offered. [step] If you haven't purchased a mattress in a while, it's good to see what options are available before going to a store. [substeps] Check prices online to see what you feel is reasonable based on what is being offered.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5967,"error":null,"output":"B"}
{"question_id":"hellaswag:00001414","item_index":363,"attempt":0,"prompt_hash":"b9ffe330da9e6fe888f9f7aeaeec50aa0355564431470bbefa6aa7966d3a478f","question":"A carved pumpking with a light in it glows on a counter. Supplies for carving are then shown. A woman cuts the top off the pumpkin, emptying the seeds. she\n\nA. removes the pumpkin and pops the top back on.\nB. cuts down all the pieces and dumps them in a trash bin in the end.\nC. then carves the traced lines to cut out the design.\nD. tapes the top shut as the continue carving the pumpkin.","prompt":"A carved pumpking with a light in it glows on a counter. Supplies for carving are then shown. A woman cuts the top off the pumpkin, emptying the seeds. she","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5966,"error":null,"output":"A"}
{"question_id":"hellaswag:00004489","item_index":364,"attempt":0,"prompt_hash":"a90d15ea2b2f0969e0a3f6585a2c7a14931973fca2f9f2f2db979fb083f53251","question":"[header] How to make homemade shower gel [title] Pour the castile soap and honey into a clean squeeze bottle. [step] Insert a funnel into the mouth of the bottle to prevent spills when you pour. Cleaned out soap, shampoo, and other bottles with easy-pour lids are ideal for this.\n\nA. [substeps] Plain) 24 ounce (162.7 ml) baby or formula bottle with rubber lining [title] Close the bottle and add water and stir. [step] About two to three minutes after adding the water and honey, close the bottle tightly.\nB. [substeps] Make sure the funnel is big enough to fit your fingers. [title] Use the bottle from the dollar store.\nC. You can use any brand or scent of castile soap you like, including : [substeps] Pure (mild) unscented floral peppermint citrus [title] Add the carrier oil. [step] With the funnel still in the mouth of the bottle, pour in the carrier oil.\nD. If you used non-soap in the shower gel recipe, you can handle the loose appearance of the bottle by adding some baby powder or moisturizing product or lip gloss. [title] Add ½ teaspoon (1/2 cup) of baking soda and a pinch of salt to the bottle.","prompt":"[header] How to make homemade shower gel [title] Pour the castile soap and honey into a clean squeeze bottle. [step] Insert a funnel into the mouth of the bottle to prevent spills when you pour. Cleaned out soap, shampoo, and other bottles with easy-pour lids are ideal for this.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5966,"error":null,"output":"C"}
{"question_id":"hellaswag:00009517","item_index":365,"attempt":0,"prompt_hash":"c9144355b6e30dae377d1b08a11277a125f014e8af198efd4f942ebe66d64435","question":"[header] How to dye hair two colors [title] Separate your hair. [step] Brush or comb your hair into two loose pigtails. This will make it easier to wrap in foil after you apply the bleach and the dye.\n\nA. Pin this section of your hair up and leave the bottom section bare. [substeps] Your clothes will be protected from the dye by the foil.\nB. [title] Find a booth in a room with air circulation. [step] If possible, it's best if you can see the entire workspace, including the counters near the tools and appliances that are often used to collect hair-bleaching products.\nC. Tie an elastic band on each segment to mark the bottom 2/3 of your hair. [title] Bleach the area below the elastic bands.\nD. Put two braids into one pigtails, one near the nape of your neck and the other behind the ears. [substeps] Each braid takes up the space between the twists and the top of your head (called the crown section).","prompt":"[header] How to dye hair two colors [title] Separate your hair. [step] Brush or comb your hair into two loose pigtails. This will make it easier to wrap in foil after you apply the bleach and the dye.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5963,"error":null,"output":"A"}
{"question_id":"hellaswag:00003988","item_index":366,"attempt":0,"prompt_hash":"c78de985a9b5cbb3c995e291eaf7425453892981bf6663aa95eb13de417aa97b","question":"[header] How to accomplish more with your master list [title] Create your master list-this is simply a brain dump of everything you want/have to do. [step] It can contain 20 , 30 , 50 + items. [title] Create your daily' to-do' list.\n\nA. [step] For example, every morning start your day with coffee, your breakfast, lunch, and maybe snacks. Skip this in later days if you can because its hard to read instead of looking at your list, or is too much of a challenge.\nB. [step] Write down a list of things you're going to have to do on your master list each day, including school, work, the kids visit, chores, plans for the upcoming month, etc. To help you think of each day ahead of time, write or look up associated goals.\nC. [step] Examples of days should come first and days last. For example, you might have monday by 12:30am, tuesday by 2:45pm, and friday by 5:00 pm.\nD. [step] This is the list of things you want to achieve today. Items on this list may come from your master list or may be a result of events from the previous day.","prompt":"[header] How to accomplish more with your master list [title] Create your master list-this is simply a brain dump of everything you want/have to do. [step] It can contain 20 , 30 , 50 + items. [title] Create your daily' to-do' list.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5963,"error":null,"output":"D"}
{"question_id":"hellaswag:00000483","item_index":367,"attempt":0,"prompt_hash":"68c0b61b1cfd79191a87dbb759f7b2aa7544f16a51b54f4d736936a9a5d14775","question":"A man is wearing blue scrubs and a fanny pack. he\n\nA. is washing his hands in the sink.\nB. begins to shave his legs with shaving cream.\nC. unzips the fanny pack and pulls it out onto a stump.\nD. is attached to a machine that is attached to his leg by a hose.","prompt":"A man is wearing blue scrubs and a fanny pack. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5964,"error":null,"output":"A"}
{"question_id":"hellaswag:00005841","item_index":368,"attempt":0,"prompt_hash":"efd2b1df79976b6e433bb353545be62fb89f63fafd62e0cab8a93b3f9a9c24c4","question":"[header] How to write a descriptive paragraph [title] Make sure to choose a meaningful person, place, or thing. [step] Before you begin your paragraph, you have to be sure to pick something that is worth describing. If you want to describe a character, make sure the person is interesting in some way.\n\nA. For example, if you would like to describe the movie you watch, make sure to choose someone who to be important to you. This can be a relative or close friend, or an acquaintance.\nB. If you don't, then the reader has no idea what it is about them that makes them interesting. [substeps] For example, a strong boyfriend, girlfriend, or partner probably wouldn't fall for an extrovert.\nC. If you want to describe an object, make sure it has some deeper meaning beyond itself so you have enough to write about. If you pick a place, make sure you can describe it in a unique way that can capture a reader's attention.\nD. For example, maybe you have interested a young child. However, if the boy is thin, it's okay to say that he's tall but still stands out, and that was why you drew him in.","prompt":"[header] How to write a descriptive paragraph [title] Make sure to choose a meaningful person, place, or thing. [step] Before you begin your paragraph, you have to be sure to pick something that is worth describing. If you want to describe a character, make sure the person is interesting in some way.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4236,"error":null,"output":"C"}
{"question_id":"hellaswag:00009900","item_index":369,"attempt":0,"prompt_hash":"20ca1a6939878931601d1991e95d3f93bd8e2169798126e081b97249991c382d","question":"[header] How to travel to kuala kapuas, central kalimantan, indonesia [title] If you want to rent a car from palangka raya to kuala kapuas, the price is around rp 350. 000. [title] One car can accommodate 7 people. [title] You can ask to the driver to take you off at the hotel in kuala kapuas.\n\nA. [title] You might relax in the car overnight and eat at the restaurant or take a short ride with the driver. [title] If you want to take a plane, ask to land at 13 airport terminals as these are sometimes pretty busy.\nB. [title] From the airport, you can ask the local taxi driver to take you to bundaran burung local terminal. [title] Price of ticket from palangka raya to kuala kapuas is around rp 20.\nC. [step] If you have a car, do not fly the afternoon aeronaureansun international airport. [title] For an extra trip, you can rent one during the daytime, during middle hours, for several hours.\nD. [title] The road should be dark and accessible. [title] It is an excellent place to walk around kuala kapuas so you don't have to worry about seeing anyone.","prompt":"[header] How to travel to kuala kapuas, central kalimantan, indonesia [title] If you want to rent a car from palangka raya to kuala kapuas, the price is around rp 350. 000. [title] One car can accommodate 7 people. [title] You can ask to the driver to take you off at the hotel in kuala kapuas.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4236,"error":null,"output":"B"}
{"question_id":"hellaswag:00003052","item_index":370,"attempt":0,"prompt_hash":"395928d246162cc17031c73fbec13809429d1823bc4a274584b2aa4e4af4cb16","question":"Slices of mangos and lemons are shown. a woman\n\nA. places how she made the lemon mold, then the dough and baking powder.\nB. takes an ax to a split log of wood on the ground.\nC. turns it around using her hand.\nD. then cuts mangos and lemons.","prompt":"Slices of mangos and lemons are shown. a woman","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4233,"error":null,"output":"D"}