{"question_id":"hellaswag:00007797","item_index":0,"attempt":0,"prompt_hash":"ca10ca786b1b6a92bc9ffaec914563572a733c03827c9b8c1642cb7ce9d7d394","question":"[header] How to save money renting a car in the united states [title] Check online travel sites. [step] You can compare car rental prices online through most travel aggregators, such as orbitz, priceline, expedia, travelocity, and kayak. Car rental prices can fluctuate wildly, so check multiple sites regularly, starting at least several weeks before you need the car.\n\nA. [substeps] If you combine your car rental with a flight and/or hotel booking, you can often save significantly more than by reserving them separately. [title] Compare prices directly from the major company websites.\nB. [substeps] Paying online usually involves submitting a bill to the online car rental service. Check car websites and discuss customizable payment plans before finalizing the transaction.\nC. [title] Sign up with a car sharing company. [step] Certain companies such as airtrain, lyft, lambordia, or banal are also carriers for car rental companies.\nD. [title] Add up your driving and parking costs. [step] Be prepared to get a loan or borrow money from a friend or family member.","prompt":"[header] How to save money renting a car in the united states [title] Check online travel sites. [step] You can compare car rental prices online through most travel aggregators, such as orbitz, priceline, expedia, travelocity, and kayak. Car rental prices can fluctuate wildly, so check multiple sites regularly, starting at least several weeks before you need the car.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3873,"error":null,"output":"A"}
{"question_id":"hellaswag:00002368","item_index":1,"attempt":0,"prompt_hash":"e64052a5f5d3d78bcd89f25163d4450867dce64feab5bcae08aabf0e15ca3f9f","question":"A woman is shown speaking to the camera while holding a dart. pictures of a man\n\nA. 's arms are shown as well as this dart.\nB. with a bowflex in one ear and a dart in her other ear next to a dart board in a room.\nC. playing darts appear and he is shaking in distress before returning to talking to the camera.\nD. are shown and leads into her throwing darts.","prompt":"A woman is shown speaking to the camera while holding a dart. pictures of a man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3884,"error":null,"output":"D"}
{"question_id":"hellaswag:00003525","item_index":2,"attempt":0,"prompt_hash":"2637f8600bc9ed564053f1df6a3c2af52481913e08cd0a17188b573318174af8","question":"[header] How to wear a bomber jacket [title] Consider fit. [step] Because they started as a utilitarian piece for servicemen strapped with equipment, they're naturally oversized. However, designers haven't been afraid to play around with the bomber jacket design.\n\nA. You've got tons of options-from thick american blazers to ¾-1-come-out bomber jackets. You just need to try them on and see if one fits you perfectly.\nB. In fact, they've made many custom bomber jackets fit for military situations in the past. [substeps] Bomber jackets are wide-brimmed, functional, and tend to be less oversized.\nC. You can wear them as an oversized piece, but you can also find them in slim fitting versions, cropped versions, and so on. [substeps] Getting the proper fit is key to rocking this look.\nD. Try a simple button-down and it can work great. You can use a plain button-down instead if you want to wear some seasonal \" steampunk \" materials.","prompt":"[header] How to wear a bomber jacket [title] Consider fit. [step] Because they started as a utilitarian piece for servicemen strapped with equipment, they're naturally oversized. However, designers haven't been afraid to play around with the bomber jacket design.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3888,"error":null,"output":"C"}
{"question_id":"hellaswag:00001285","item_index":3,"attempt":0,"prompt_hash":"30d7069fbc34f77271c8d0b8e835a09d88aedf5faac000da1f602802d19f3f28","question":"The players stand around talking to one another and one shoots a bow and arrow at a target. more people\n\nA. watch while the owners wait.\nB. are seen shooting the bows followed by interviewers speaking and team mates reacting.\nC. walk by and the players talk to each other in the background.\nD. come and go behind the referee who has just come on stage to take picture and record the game.","prompt":"The players stand around talking to one another and one shoots a bow and arrow at a target. more people","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6764,"error":null,"output":"D"}
{"question_id":"hellaswag:00003679","item_index":4,"attempt":0,"prompt_hash":"1d6c41ac9865073b0c0a892c7f71f04af82decb25d8331a4c1e7d911ffc5e84f","question":"[header] How to clean hairbrushes [title] Remove the hair from the brush. [step] Taking out the old hair is always a good idea, since dirt and skin particles tend to stick to it as it builds up in your brush. Reach in and pinch the hair with your fingers, then pull it out from around the bristles.\n\nA. [substeps] Brushes that are attached to anything (such as the brush) will be clean and free of dirt and bacteria, but brushes with metal can damage them and produce residue. [title] Choose one that is easy to clean.\nB. [substeps] These small pinches are usually easy to remove or you can simply take them out. Removing used hair from an old hairbrush, just may take some effort to get it off of the handle.\nC. Keep doing this until you've extracted all of the hair from the brush, then throw the hair away. [substeps] If your brush's bristles are made from natural fibers, take care not to yank them out with the hair.\nD. [substeps] Most brushes have a teeth crown screw or other attachment. If not, put the brush in to remove the crown.","prompt":"[header] How to clean hairbrushes [title] Remove the hair from the brush. [step] Taking out the old hair is always a good idea, since dirt and skin particles tend to stick to it as it builds up in your brush. Reach in and pinch the hair with your fingers, then pull it out from around the bristles.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3880,"error":null,"output":"C"}
{"question_id":"hellaswag:00002729","item_index":5,"attempt":0,"prompt_hash":"25a7613dfe9e26f0695e306fabcf80652f3ef7ae0ed559a6506fe89c64ac1096","question":"A clip begins and it features people running and the words on the screen say they are novices who are training to run a marathon in tel aviv, israel. a man\n\nA. is seen speaking in a temple.\nB. is then shown running and the words say can be seen on the screen again saying he has some good news.\nC. appears training them and different marathoners in training take turns being interviewed while they continue to switch back and forth to everyone going through the training.\nD. is seen standing in front of a street and he begins to perform an act while holding the statue and flipping back and fourth.","prompt":"A clip begins and it features people running and the words on the screen say they are novices who are training to run a marathon in tel aviv, israel. a man","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3886,"error":null,"output":"B"}
{"question_id":"hellaswag:00005557","item_index":6,"attempt":0,"prompt_hash":"fd9e38b8c08f28b65a475a7d1c028ed7398abfaa377a7eff9a603a839d73cdb7","question":"[header] How to tie a doo rag [title] Place the durag on your head. [step] You can choose a color and style of durag that work for you. Many people prefer doo rag that are a stretchy material that can be seen through when stretched.\n\nA. They look like long cargo pants or long jacket pieces. Your crotches will have to be about even or slightly larger than the durag's elastic.\nB. This makes them more breathable when they're tied tightly around your head. [substeps] Line the center seam up with the center of your head for symmetry.\nC. The durag should be loose enough so that you can loosely wrap the rag around your head. [substeps] Your durag may be hard to place with one hand or with two hands so you need to make sure that it will allow you to handle it.\nD. Use a product like dawww.dahoo.com or zieforednwearer if you want a casual cut. Durag fabric is also often called sewn cloth and is woven with hemp or microfiber.","prompt":"[header] How to tie a doo rag [title] Place the durag on your head. [step] You can choose a color and style of durag that work for you. Many people prefer doo rag that are a stretchy material that can be seen through when stretched.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3870,"error":null,"output":"B"}
{"question_id":"hellaswag:00002795","item_index":7,"attempt":0,"prompt_hash":"dd25d87c8fb2a9b5be52be8446c91919c0b7290b09e1f1eaa2de509920660f63","question":"A towel and glass are on a table next to a piece of foam and metal. A tool is being shown by a man holding it. the metal\n\nA. is cooled into a liquid and garnishes the fencing material.\nB. is separated from the foam and put over a bath tub before being pressed with sponge and rinsed off.\nC. is used to sharpen a knife in front of him.\nD. is being used on a piece of cloth, cloth, toothbrush and shirt.","prompt":"A towel and glass are on a table next to a piece of foam and metal. A tool is being shown by a man holding it. the metal","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4311,"error":null,"output":"A"}
{"question_id":"hellaswag:00000102","item_index":8,"attempt":0,"prompt_hash":"5a11413033d28b4e1419bacb3b0d5fb6df3cdf32ca0da43168f68740af83539a","question":"A man is in the gym in tight he bends over picks up a weight over his head and drops it back down. he\n\nA. lifts his legs and pulls the bar up on his body and lifts the weight above his head.\nB. lifts it again lifts it over his head and drops it this time and he loosens his way to the top and grabs it again.\nC. raises the bar over his head followed by another.\nD. walks back and loosens up before walking back up and doing it again adding more weight.","prompt":"A man is in the gym in tight he bends over picks up a weight over his head and drops it back down. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3884,"error":null,"output":"D"}
{"question_id":"hellaswag:00004664","item_index":9,"attempt":0,"prompt_hash":"9bb2edc6249671d4af6b2ade6a4c2317652eac6fccb9834c5a04cc81e5791c5d","question":"[header] How to care for hair between salon trips [title] Buy a deep conditioner. [step] You probably already have shampoo and conditioner. The conditioner you use regularly does condition your hair as thoroughly as a deep conditioner.\n\nA. A deep conditioner will make your hair appear softer, shiner, and healthier. It should only be used once a week.\nB. However, you're willing to risk starting your hair oily or looking like a buck without a deep conditioner. Look for a deep conditioner made specifically for you hair at your local salon.\nC. But you don't want to purchase a deep conditioner as any hair at all. Invest in a deep conditioner and make sure you use it daily.\nD. Choose a deep conditioner, spritz it on your hair, then shampoo, then blow dry. You can simply apply the deep conditioner once a day and then continue rinsing to replace it over time.","prompt":"[header] How to care for hair between salon trips [title] Buy a deep conditioner. [step] You probably already have shampoo and conditioner. The conditioner you use regularly does condition your hair as thoroughly as a deep conditioner.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3887,"error":null,"output":"A"}
{"question_id":"hellaswag:00001467","item_index":10,"attempt":0,"prompt_hash":"5b7790382de9fbd49f3d4860bb12b2f5e34e6ffe7911d0c0c38609426d5167b7","question":"A girl runs down a track in slow motion. she\n\nA. is throwing a javelin over the bar.\nB. smashes into a sand pit several times.\nC. takes a huge jump, hurling herself over a bar.\nD. jumps into a sand box and begins to play a game of hopscotch.","prompt":"A girl runs down a track in slow motion. she","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3883,"error":null,"output":"A"}
{"question_id":"hellaswag:00006499","item_index":11,"attempt":0,"prompt_hash":"a30d372168a2e9ad0fc7e0ffffaaceffa522c9dc15224a6d6dc8a261da673f6a","question":"[header] How to make your hair stay straight all day [title] Use shampoos and conditioners that help straighten hair. [step] Look for shampoos and conditioners that have lots of vitamins and minerals in them, giving your hair refreshing nutrients that will help smooth it out. These are usually marketed as smoothing, sleek, or straightening products.\n\nA. [substeps] Look for a shampoo that lists \" hydrating ingredients \", which are sodium and other all natural ingredients. You can also try going for one that specifically states \" hydrating \" on the bottle.\nB. [substeps] If you have very curly hair, you may need to consider using a straightening shampoo. You can also purchase a hair-straightening powder and conditioners to help moisturize your hair.\nC. If you have dry hair, look for shampoos that will keep it soft, wet, and shine. If you have hair that is frizzy, opt for sleek, volumizing, or foam shampoos.\nD. [substeps] Look for ingredients such as wheat proteins, pro vitamin b5, or hedera helix extract. Take the time to comb the conditioner into your hair while in the shower, using your fingers or a wide-toothed comb.","prompt":"[header] How to make your hair stay straight all day [title] Use shampoos and conditioners that help straighten hair. [step] Look for shampoos and conditioners that have lots of vitamins and minerals in them, giving your hair refreshing nutrients that will help smooth it out. These are usually marketed as smoothing, sleek, or straightening products.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3870,"error":null,"output":"B"}
{"question_id":"hellaswag:00006334","item_index":12,"attempt":0,"prompt_hash":"00e21ab40d9fe71e8c1fe137f6ba5f3eb8dd787e55e94031752ced702c320e72","question":"[header] How to take care of black girls' hair [title] Wash your hair once every 1 to 2 weeks. [step] The more often you wash your hair, the more you strip it of its moisture. This results in dry, brittle hair.\n\nA. Shampoo your hair only once every 2-3 days, and use a rinsing power for the shortest-lasting results. [title] Rinse out your hair with cool water, then towel it dry.\nB. [substeps] Get a conditioner applied after every one to two washes. Use an spritz of hair spray.\nC. [substeps] If you have moderately-oily hair, you may need to shampoo more often in the time after you wash it. As a general rule, you should shampoo at least once every 2 weeks (if not more).\nD. You should wash your hair every 7 to 10 days, but you can get away with washing it once every 14 days. This will help prevent dryness and product buildup.","prompt":"[header] How to take care of black girls' hair [title] Wash your hair once every 1 to 2 weeks. [step] The more often you wash your hair, the more you strip it of its moisture. This results in dry, brittle hair.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3875,"error":null,"output":"D"}
{"question_id":"hellaswag:00006287","item_index":13,"attempt":0,"prompt_hash":"7bc8b53a23b4caf4992746012d1db77af431bed164c545dcab89d3b7c19de023","question":"[header] How to make razor blades last longer [title] Clean the razor after each use. [step] Salt from your skin will corrode your razor blade, causing it to dull faster. Don't stop at simply rinsing the blade, give it a good scrubbing.\n\nA. The blade shouldn't feel gritty or like sand. Rinse the blade between frequently dirty brushes.\nB. Using an old toothbrush, simply scrub across the blade a few times to clean away any leftover skin and hair, before rinsing it in water again. [title] Dry your razor thoroughly after each use.\nC. You shouldn't go farther than a quarter of an inch or so, but you can go a fraction farther. [title] Moisten the blade with hydrogen peroxide.\nD. Scrub down the blade of the razor with a hand towel as soon as you see any nicks or scratches. Rinse your razor with running water if you see anything once you've scrubbed it.","prompt":"[header] How to make razor blades last longer [title] Clean the razor after each use. [step] Salt from your skin will corrode your razor blade, causing it to dull faster. Don't stop at simply rinsing the blade, give it a good scrubbing.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3876,"error":null,"output":"B"}
{"question_id":"hellaswag:00001181","item_index":14,"attempt":0,"prompt_hash":"b54501e55364b34bce298493cd27b95c3d6415beacc3d1b626d6ea661c332b29","question":"This woman rides the white/brown horse under the shady trees for a little bit. the horse\n\nA. is nodding its head and walks in circles a few times.\nB. follows the woman under the oaks.\nC. walks beside the woman letting her go for a little while.\nD. comes to a stop and then the woman with the red tack begins to brush the horse's mane.","prompt":"This woman rides the white/brown horse under the shady trees for a little bit. the horse","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6762,"error":null,"output":"D"}
{"question_id":"hellaswag:00009199","item_index":15,"attempt":0,"prompt_hash":"5831059f5d4a6c72dff8dc8e656b9176e8ed3762579dad7e5accfca901734a00","question":"[header] How to buy teacher supplies [title] Communicate with your supervisor. [step] A purchase order lets your school know what you're buying. Most schools require you to request a purchase order before shopping, especially if your school requires you to buy from specific stores.\n\nA. Always, have a written record of the purchases you buy. [substeps] Your school should be able to supply you with a receipt of the purchase order.\nB. Contact a teacher directly at or in your school and tell them you want the purchase order. [title] Talk to your school's principal, if there is one.\nC. Ask your school's bookkeeper if a purchase order is required before you start buying supplies. You'll need to write down each purchase on the order and return it to your school when you're finished buying your supplies.\nD. If you're unsure about what you can buy, ask your teacher. Depending on your school, you may also get help from a principal.","prompt":"[header] How to buy teacher supplies [title] Communicate with your supervisor. [step] A purchase order lets your school know what you're buying. Most schools require you to request a purchase order before shopping, especially if your school requires you to buy from specific stores.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3888,"error":null,"output":"C"}
{"question_id":"hellaswag:00006899","item_index":16,"attempt":0,"prompt_hash":"00b1132ce0254dc707fd25cf3e98e54f1ff08368fd442e193d638defc7dcf05e","question":"[header] How to make your room look cool [title] Lay down or display shag rugs. [step] A shag rug can be a nice and soft addition to your room's floor but you may also nail it up on a wall for a unique display. Choose a rug with a unique pattern so that it draws the eye.\n\nA. [title] Add cool pillows to your bed or couch. [step] You may be able to find pillows with interesting quotes, pictures, designs and textures to add to your room.\nB. For example, if you have a fleece style rug, use that as your display. Make sure the pattern matches your bedspread.\nC. [substeps] If you can, choose a style you will generally love. Instead of traditional wall rugs, choose patterned rugs or texture rugs to accent your room.\nD. You may have a small flat rug in the bedroom, or move some to a couch in the living room. [title] Go all out with a field rug.","prompt":"[header] How to make your room look cool [title] Lay down or display shag rugs. [step] A shag rug can be a nice and soft addition to your room's floor but you may also nail it up on a wall for a unique display. Choose a rug with a unique pattern so that it draws the eye.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6762,"error":null,"output":"A"}
{"question_id":"hellaswag:00006824","item_index":17,"attempt":0,"prompt_hash":"33fdf255c97842163036cb29bde1c9ea7547b2f868bb4ea53e44801e858d8296","question":"[header] How to cook broccolini [title] Choose bright green broccolini. [step] Fresh broccolini will also have fresh bases, firm stems, and tightly furled heads. [substeps] If the florets are beginning to turn yellow, the broccolini is no longer good.\n\nA. Canned or dried broccolini should still be excellent, as they have a bright green stem and firm leaves. However, white broccolini can only have mild to moderate flavor.\nB. The same applies if the bases appear dried out, if the stems have soft spots, or if there are signs of flowering on the heads. Broccolini should be stored in a tightly sealed plastic bag or container inside the refrigerator until ready to use.\nC. If the broccolini is on its last layer of leaves, it is still good, but not yet ready to make some export steamed bread. Broccolini will have more mayonnaise than the finely milled kind, so you may think it's more appropriate to cook it instead.\nD. Cook the broccolini in the refrigerator. The firm, bright blooms will take the broccolini out of the refrigerator once it is no longer fresh.","prompt":"[header] How to cook broccolini [title] Choose bright green broccolini. [step] Fresh broccolini will also have fresh bases, firm stems, and tightly furled heads. [substeps] If the florets are beginning to turn yellow, the broccolini is no longer good.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3879,"error":null,"output":"B"}
{"question_id":"hellaswag:00005730","item_index":18,"attempt":0,"prompt_hash":"b7cf26219347b5962b66ac08eb85520e9824095fb2a51be21ebda5e9b6951f74","question":"[header] How to become a cartographer [title] Understand the job description. [step] All cartographers must be able to visualize data using both analysis and artistic imagination. They use a close attention to detail and accuracy in order to translate the data into a map, and have a reputation for perfectionism.\n\nA. In general, cartographers must understand how to solve problems properly. [substeps] You want to come up with a specific category for your current cartography talents, then put these skills into action.\nB. Sketchacosee is a master cartographer, who uses these qualities to develop and arrange detail, and has a creative mind that critiques images, scenes, and ideas. Whenever you're researching how to become a cartographer you'll have to pay attention to how people would view your writings.\nC. Computer programming skills and mathematical ability are increasingly valued in the cartography job market, but you do not necessarily need advanced experience in these subjects. [substeps] Most cartographers have office jobs, and spend 35-40 hours a week working mostly on the computer and drafting table.\nD. For cartographers of all ages, the career paths, licensing, and skills are often discussed in private schools or university campuses. [title] Decide if the career path is appropriate for you.","prompt":"[header] How to become a cartographer [title] Understand the job description. [step] All cartographers must be able to visualize data using both analysis and artistic imagination. They use a close attention to detail and accuracy in order to translate the data into a map, and have a reputation for perfectionism.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3872,"error":null,"output":"C"}
{"question_id":"hellaswag:00001983","item_index":19,"attempt":0,"prompt_hash":"22f8da56324f138171b9de68b3ac6a97b6278eba1974d3e3ec572efc08dc54a5","question":"The lady puts contents of a bottle in her palms, and the lady rubs it on her face. The lady makes funny faces. The lady washes her face. The lady towel dries her face. the lady\n\nA. grab a tissue and wipe her face.\nB. puts spots of white lotion on her face and rubs it in.\nC. puts a towel in her eyes.\nD. holds a glass of water and talks.","prompt":"The lady puts contents of a bottle in her palms, and the lady rubs it on her face. The lady makes funny faces. The lady washes her face. The lady towel dries her face. the lady","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6762,"error":null,"output":"B"}
{"question_id":"hellaswag:00007078","item_index":20,"attempt":0,"prompt_hash":"2f21ca3e374d6d98bc2f0fad6175ee0e2440b403da161e88b764ed660b16661f","question":"[header] How to grow your hair out for your wedding day [title] Eat healthy fats and proteins. [step] One important factor in growing your hair out is your diet, so make sure to incorporate plenty of healthy fats, omega 3 fatty acids, b vitamins and proteins into your daily meals. Foods rich in these nutrients include salmon, avocados, fatty fish and nuts.\n\nA. [substeps] There are many foods to choose from. Sweet asparagus, red meat, turnips and artichokes are all good sources of omega 1 fats.\nB. [substeps] Breakfast should be one small meal with plenty of lean meat or eggs. Lunch should be mostly fruit and carbohydrates.\nC. [substeps] Studies have shown that regular and steady growth lead to a naturally long hair appearance. [title] Use a quality shampoo, conditioner and hair mask to smooth and tame your hair.\nD. [substeps] You will need to keep up this diet for a few months to see results, so be sure to start well before the month of your wedding. Since hair grows at different rates, you should start at least 6 months in advance.","prompt":"[header] How to grow your hair out for your wedding day [title] Eat healthy fats and proteins. [step] One important factor in growing your hair out is your diet, so make sure to incorporate plenty of healthy fats, omega 3 fatty acids, b vitamins and proteins into your daily meals. Foods rich in these nutrients include salmon, avocados, fatty fish and nuts.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3878,"error":null,"output":"D"}
{"question_id":"hellaswag:00007915","item_index":21,"attempt":0,"prompt_hash":"342258f10ed11139fdfbd2b866a6b43ed45e82d21dc5c78d14556bfbc13ca5cc","question":"[header] How to install chain link fence [title] Obtain any necessary permits. [step] Your local government may have building and zoning regulations that regulate fence setbacks, type, and height. If you install a fence without a permit, they may tear down the structure.\n\nA. [title] Establish where your property lines are located. [step] This information can be obtained from city records, a realtor's line plot map, or by hiring a surveyor.\nB. [substeps] If you get a construction permit, then you cannot attempt to install a fence until the building, zoning, and development process is underway. Even if there are no zoning restrictions, this can result from fines and/or foreclosure.\nC. [title] Visit a website that will allow you to buy the type of fence you want to install. [step] You will need to find a particularly attractive and well-maintained site to start your chain link fence.\nD. The required permits will differ from state to state. [substeps] Visit the national park service website to find out which permits are required in your state.","prompt":"[header] How to install chain link fence [title] Obtain any necessary permits. [step] Your local government may have building and zoning regulations that regulate fence setbacks, type, and height. If you install a fence without a permit, they may tear down the structure.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3876,"error":null,"output":"D"}
{"question_id":"hellaswag:00007552","item_index":22,"attempt":0,"prompt_hash":"f9f3485276d639673d5adfd98d8250b82e09f9cab2bf9aa3b85ffccffd9e29ee","question":"[header] How to leave friends when moving (teenagers ) [title] Journal out your thoughts. [step] Journaling is a great tool if you want to work through the emotions that you're experiencing. Get out some paper and a pencil and write out your thoughts, however they come to you.\n\nA. This will make writing them easier to remember and freer you to think clearly about what's going on in your head. You will definitely remember how this experience made you feel.\nB. Try writing everything down, from your name to your cat's name to the name of a fictional character. Whatever's on your mind is apt to come to you.\nC. You can even imagine them as a pin on your bun or the first page of your text text mail. [substeps] Write down what you're feeling and who exactly you're writing it to.\nD. [substeps] You might express yourself best through poetry or songwriting. You can also try writing out a list of all the worries you have about this new move and what you can do about those worries.","prompt":"[header] How to leave friends when moving (teenagers ) [title] Journal out your thoughts. [step] Journaling is a great tool if you want to work through the emotions that you're experiencing. Get out some paper and a pencil and write out your thoughts, however they come to you.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3878,"error":null,"output":"D"}
{"question_id":"hellaswag:00001060","item_index":23,"attempt":0,"prompt_hash":"2399bcbc0fc55cc358a57ffecbe1d33fda15ae1d77bc104d2d4e40f8eadba317","question":"A man in a white shirt is holding a tool in his hand. He starts chopping weeds down in a field. he\n\nA. is painting a fence in the field.\nB. stops and puts the tool against a fence.\nC. moves the tool to cut down a tree.\nD. removes the spiky weeds off with the tool.","prompt":"A man in a white shirt is holding a tool in his hand. He starts chopping weeds down in a field. he","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3883,"error":null,"output":"B"}
{"question_id":"hellaswag:00001051","item_index":24,"attempt":0,"prompt_hash":"60c8857cf55aa85e2a87f883c8ddf9c355ecad336ce96dcb9b372a66cdf9799d","question":"A man is standing under a basketball goal. A group of boys is listening to the coach. they\n\nA. pass by the goal.\nB. switch places and take turns.\nC. begin warming up by doing lay ups.\nD. are acting like boys did that basketball performance.","prompt":"A man is standing under a basketball goal. A group of boys is listening to the coach. they","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3886,"error":null,"output":"B"}
{"question_id":"hellaswag:00000298","item_index":25,"attempt":0,"prompt_hash":"03d84bbc15735cfa564aca31186da43e25256ffe4af36768d209b36c8d1d7636","question":"A man is standing in a field before running and jumping into a square of dirt. He walks away from the jump and talks to the camera. it\n\nA. appears to its red and green color, which brings the man back on the field.\nB. ends and he claps his hands over his mouth.\nC. is shown again in slow motion.\nD. changes to him jumping again in the same spot.","prompt":"A man is standing in a field before running and jumping into a square of dirt. He walks away from the jump and talks to the camera. it","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3885,"error":null,"output":"C"}
{"question_id":"hellaswag:00007414","item_index":26,"attempt":0,"prompt_hash":"9dfc210608de98057c733904f47301a039e0cf1bdd73fc48a03d23c5d4a08c0a","question":"[header] How to pucker your lips [title] Relax your lips. [step] Before puckering your lips, keep them relaxed. Let your lips hang naturally as this will keep them softer and more gentle during kissing.\n\nA. [title] Place your index and middle fingers slightly apart from each other. [step] Your pinkie and ring fingers should be located with your middle and middle fingers.\nB. [title] Practice forming your breath as you inhale. [step] It's easier to pucker your lips if you physically move them slightly downward.\nC. Just before a kiss, pay attention to your lips and make sure you're not tensing them up. [substeps] To help relax your lips, you can breathe through your mouth slightly to loosen up your lips.\nD. [title] Keep your lips on the inside of your mouth. [step] Be gentle; too much lip-puckering can result in prolonged open up and blowing bubbles.","prompt":"[header] How to pucker your lips [title] Relax your lips. [step] Before puckering your lips, keep them relaxed. Let your lips hang naturally as this will keep them softer and more gentle during kissing.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3885,"error":null,"output":"C"}
{"question_id":"hellaswag:00010033","item_index":27,"attempt":0,"prompt_hash":"f89c9edd2fdf2b77469e72c76f55d29a799cb5293fea554b170e2b79d32cd109","question":"[header] How to wear white shoes [title] Choose low top over high top white sneakers for versatility. [step] Low tops look good with a wide range of outfits, so you'll get the most mileage out of them. Their versatility makes them more appropriate than high tops in many situations and settings.\n\nA. You also won't have to deal with your pants bunching up at the ankles when wearing low tops. [substeps] Go with a pair of classic trainers if you wear jeans and t-shirts often.\nB. Here are some trends to keep in mind when choosing low top white sneakers : [substeps] If you're walking around town, try some dowdy ankle-high sandals. High tops are those with small straps on either side and flare out at the edges.\nC. If you're comfortable wearing a pair of high top white sneakers, pair them with white jeans or other pastel clothing. Look for the one with low bottoms, as they'll look super stylish without the sneakers.\nD. [substeps] Pair high top white sneakers with denim shorts, an oversized cable top, or a crew-neck tee. [title] Pair high top white sneakers with classic a-line pants for outfits made up of dark pants and boots.","prompt":"[header] How to wear white shoes [title] Choose low top over high top white sneakers for versatility. [step] Low tops look good with a wide range of outfits, so you'll get the most mileage out of them. Their versatility makes them more appropriate than high tops in many situations and settings.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6763,"error":null,"output":"A"}
{"question_id":"hellaswag:00009120","item_index":28,"attempt":0,"prompt_hash":"462ce44a5d1f0c59b3f207a6375f0dc2a1fc15436b6b61322c00caef198066f6","question":"[header] How to color your hair without using hair dye [title] Use brewed coffee and conditioner to darken your hair. [step] Place 2 cups (470 ml) of leave-in conditioner into a bowl. Stir in 2 tablespoons (10 g) of coffee grounds and 1 cup (240 ml) of room-temperature brewed coffee.\n\nA. Brew the mixture for 2 minutes until the coffee thickens. [substeps] If you don't have brewed coffee yet, substitute 1-2 teaspoons (10 ml) of powdered coffee grounds.\nB. Apply the mixture to your hair, wait 1 hour, then rinse it out with warm water. To help set the color, rinse your hair with apple cider vinegar afterward, then let your hair air dry.\nC. The coffee will absorb the dye, creating a hair-blonde shade. [substeps] Coffee grounds will be more likely to dull your hair if you use colored/brewed coffee.\nD. You want a very hot surface for the dye to stay on. [substeps] If you want to make your hair lighter by storing the dye on your hair for overnight, add leftover coffee grounds.","prompt":"[header] How to color your hair without using hair dye [title] Use brewed coffee and conditioner to darken your hair. [step] Place 2 cups (470 ml) of leave-in conditioner into a bowl. Stir in 2 tablespoons (10 g) of coffee grounds and 1 cup (240 ml) of room-temperature brewed coffee.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3877,"error":null,"output":"B"}
{"question_id":"hellaswag:00000592","item_index":29,"attempt":0,"prompt_hash":"9b267567d3a334fd761d5ca45dde0154182e145b156ace779bb50bba8511b017","question":"Four men play beach volleyball while serving, passing and hitting the ball. A little girl stand behind the players. then\n\nA. , the girl and the two other girls continue playing.\nB. , the players stand as spectators, while the girls run and swim.\nC. , the young man play beach volleyball with another guy.\nD. , the men change sides and continues playing beach volleyball.","prompt":"Four men play beach volleyball while serving, passing and hitting the ball. A little girl stand behind the players. then","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3884,"error":null,"output":"D"}
{"question_id":"hellaswag:00001219","item_index":30,"attempt":0,"prompt_hash":"8312bef4224256a041e57c8ce7783f3558661aba3cc9108dc61b5a060335f476","question":"A man is standing outside turning on his hedge trimmer turning it around to make the blades go. He then walks over to the hedges and begins to cut down the leaves on it. as he\n\nA. cuts it asia walks towards the lawn.\nB. is cutting by himself, a car drives by that is parked over in the yard.\nC. is cutting it, he begins to walk into the hedges to cut the hedge evenly and shaking the cut leaves off with his hand.\nD. is doing his mowing he trips and falls down.","prompt":"A man is standing outside turning on his hedge trimmer turning it around to make the blades go. He then walks over to the hedges and begins to cut down the leaves on it. as he","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3883,"error":null,"output":"C"}
{"question_id":"hellaswag:00004924","item_index":31,"attempt":0,"prompt_hash":"d293bddd91995178ae663923eab0c9430eb0886b2ff11ddf17005eff15fe6a0e","question":"[header] How to do gradient smokey eye makeup [title] Pick a base color. [step] One of the more popular base colors for a smokey eye look is black or gray. You may go for black or gray as your base if you want a more traditional smokey eye for a night out.\n\nA. Black or gray also be a good option if you are planning on wearing a colorful outfit and want your eye makeup to be more neutral and dark. Another option is to go for a base color that is more colorful, such as blue, purple, green, or pink.\nB. Alternatively, you may mix it up with other colors so that your smoky eye looks the same. [title] Mix dark eyeliner with dark eyeliner.\nC. Or, you may choose an instantly interesting shade for an eye the exact same color as your shadow color. Match the color of your base color to your shadow color to give it dimension.\nD. [substeps] Most earth tones are deep and muted. For a darker smokey eye, try pink, black, or purple.","prompt":"[header] How to do gradient smokey eye makeup [title] Pick a base color. [step] One of the more popular base colors for a smokey eye look is black or gray. You may go for black or gray as your base if you want a more traditional smokey eye for a night out.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3887,"error":null,"output":"A"}
{"question_id":"hellaswag:00007438","item_index":32,"attempt":0,"prompt_hash":"732f3fc12c449dae09d0a9e087ae06f51ab10b41d17c69a64cf9cc7c75bbf4ed","question":"[header] How to groom a beard [title] Invest in the right tools. [step] To properly trim your beard, you'll need a wide-toothed comb, scissors and a beard trimmer. Consider purchasing a pair of professional barber's scissors for daily maintenance and trimming flyaway hairs.\n\nA. Invest in a quality beard trimmer for shaping and to trim consistent lengths. For the sake of convenience, consider buying a rechargeable cordless model.\nB. However, not all beards require barber's scissors. Consider using a razor with blades that are set to clean all hair, even the part close to the scalp.\nC. This will ensure you use the right tool when cutting, and will keep your beard clean and trim for its full length. [substeps] If you're trying to save money or don't want to cut your beard, you may want to invest in a beard trimmer.\nD. You should also find a beard trimmer that cuts your beard into even sections. [substeps] Regular trimmers are good for beginners because they are good for picking out hairs and removing skin.","prompt":"[header] How to groom a beard [title] Invest in the right tools. [step] To properly trim your beard, you'll need a wide-toothed comb, scissors and a beard trimmer. Consider purchasing a pair of professional barber's scissors for daily maintenance and trimming flyaway hairs.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3549,"error":null,"output":"A"}
{"question_id":"hellaswag:00002301","item_index":33,"attempt":0,"prompt_hash":"dd21ce1fbf7063a300294494b31ef909f60fc2adeff61f9f9ae35a44ebc1743d","question":"A group of chefs are gathered around a table. the head chef\n\nA. begins talking to the camera.\nB. is talking into a microphone about something.\nC. is demonstrating how to sharpen a knife.\nD. swings slowly at a lettuce and peeler.","prompt":"A group of chefs are gathered around a table. the head chef","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3556,"error":null,"output":"C"}
{"question_id":"hellaswag:00006076","item_index":34,"attempt":0,"prompt_hash":"ba4f0cf01c250fd0c70e2c73f7182d6ce1cd490146c876b2942044d6c4c61b66","question":"[header] How to stay fresh [title] Jump in the shower. [step] To stay fresh, start your day by cleansing your body. Your unique body chemistry will determine how often you need to bathe.\n\nA. When showering, always fill the bathtub with water. Add warm water from the faucet to make sure to add enough water to saturate your pubic hair.\nB. Warm water can last longer at first, but after that you may need to follow that up by subadding hot water to clean yourself. A warm shower can be especially important if you are currently wearing spandex.\nC. Many people take a shower every morning, but if you play sports or live in a really humid climate, twice a day might help you stay fresher. If you live in a temperate place and your skin is on the dry side, every other day might be best.\nD. Establish a routine in which you shower, taking shower or bathing in the morning or at night. [substeps] If you have a hard time incorporating a routine into your job or lifestyle, think about splitting a morning shower into several smaller days like that or setting aside a specific time for showering to sleep.","prompt":"[header] How to stay fresh [title] Jump in the shower. [step] To stay fresh, start your day by cleansing your body. Your unique body chemistry will determine how often you need to bathe.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3547,"error":null,"output":"C"}
{"question_id":"hellaswag:00006077","item_index":35,"attempt":0,"prompt_hash":"f884b28c173ff709e4799a91d5d14b2115cc10305833c0c29268b7af83496793","question":"[header] How to negotiate an interest rate on a lease buy out car loan [title] Consider simply ending your lease. [step] A buy out is just one of your options at the end of a lease. You can also just end the lease by making your final payment and returning the car.\n\nA. However, it is not usually enough to end the lease. [substeps] To end a lease with a car rental company, you have to provide that company with a check, a buyer receipt, and some proof that you have saved.\nB. If you are ready to move on or don't like the car any more, this is a perfectly reasonable way to end the lease. However, you may also choose to buy the vehicle at the end of your lease.\nC. The first requirement for an out of control down payment is not to end the lease at all; this is defined as exactly how you intend to end the lease if you fail to make payments during the lease term. [substeps] An out of control down payment happens as soon as you use the loan to get the car out.\nD. This also does not require high interest rates and will improve your chances of getting the loan faster and more effectively. If you do decide to end your lease, call your creditor or similar agency to let them know you want to end the lease.","prompt":"[header] How to negotiate an interest rate on a lease buy out car loan [title] Consider simply ending your lease. [step] A buy out is just one of your options at the end of a lease. You can also just end the lease by making your final payment and returning the car.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3547,"error":null,"output":"B"}
{"question_id":"hellaswag:00004059","item_index":36,"attempt":0,"prompt_hash":"b382f27c69f46be675ce43ca160da8bf1497464fc85c9852c7a71666c83f2d8d","question":"[header] How to draw henna tattoos [title] Learn about henna motifs. [step] Henna tattoo designs feature common images and shapes. These include flowers, tendrils, stars, vines, crisscross patterns, paisleys, spirals, water drops, and leaves.\n\nA. You can either choose to use different images of henna to enhance your drawing, or find a site that caters to your design preference. [substeps] Specific elements from individuals's home history and allegiance to a political party or group usually create more dramatic henna tattoos.\nB. These designs tend to use repeating patterns and have a flowing feel. [substeps] Use google image search to view some examples of henna tattoos and get some ideas.\nC. Henna tattoo designs will often change over time. It is important to consider and focus on the various types of henna that could be used in a custom design.\nD. These elements give the tattoo intricate, dramatic details, but aren't necessary for most touts. [substeps] This works best for poorly developed platforms like a more complex interface.","prompt":"[header] How to draw henna tattoos [title] Learn about henna motifs. [step] Henna tattoo designs feature common images and shapes. These include flowers, tendrils, stars, vines, crisscross patterns, paisleys, spirals, water drops, and leaves.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3546,"error":null,"output":"B"}
{"question_id":"hellaswag:00003781","item_index":37,"attempt":0,"prompt_hash":"8b002408c490897d1c69e2e61e46fe84eb9ecece880b8ce46eae97c75d895aa8","question":"[header] How to wear a saree in bengali style [title] Tuck the saree in your waist. [step] Start the tuck on the right side of your waist, and wrap it all the way around your body until it comes back to the right side of your body. As the edge runs along your waist, you can gently tuck in the entire edge, so the saree has a secure fit.\n\nA. [title] Use the saree throughout your workout. [step] If you want to look professional, keep the saree in place throughout your workout.\nB. [title] Pull the saree in a circle around your waist. [step] Next, look up at your back and try to take into account where the right side of your waist is.\nC. [substeps] The bottom border of the saree should graze the top of your feet, and should look horizontally straight and parallel to the ground. [title] Pleat the saree by tucking it back and forth.\nD. [substeps] This is also where the part of your saree that passes under your belly flap is referred to. [title] Gather the saree before putting on any jewelry.","prompt":"[header] How to wear a saree in bengali style [title] Tuck the saree in your waist. [step] Start the tuck on the right side of your waist, and wrap it all the way around your body until it comes back to the right side of your body. As the edge runs along your waist, you can gently tuck in the entire edge, so the saree has a secure fit.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3545,"error":null,"output":"A"}
{"question_id":"hellaswag:00008322","item_index":38,"attempt":0,"prompt_hash":"14883c35fae5924a9a9b0e9f665ecb0341851957f7ef2739e6a87e36bdd94300","question":"[header] How to be a man of the world [title] Practice good hygiene. [step] Personal hygiene is a basic part of showing others you're sophisticated. Bathe regularly, brush and floss your teeth, wear deodorant, and trim your nails.\n\nA. They should be neat, odor-free, and sharp. However, loose nails can also hurt eyes.\nB. [substeps] Choose clean, everyday clothes (and clothes that don't smell body odor) and wash your clothing at least twice per week. Wash your face, brush and floss once every other day, and use essential oil, even if it's not body-soft.\nC. If at all possible, have a dedicated figure-8-day spa day once a week. Use body spray and body oil both before and after you shower so your breath stays fresh and dewy.\nD. Try to avoid leaving the house with wrinkled, dirty clothes or messy hair. [title] Build a button-down shirt collection.","prompt":"[header] How to be a man of the world [title] Practice good hygiene. [step] Personal hygiene is a basic part of showing others you're sophisticated. Bathe regularly, brush and floss your teeth, wear deodorant, and trim your nails.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3544,"error":null,"output":"D"}
{"question_id":"hellaswag:00000934","item_index":39,"attempt":0,"prompt_hash":"85d45e891fbcea8182b731f5980611cf9c83a88f81b6795cea08812b6cc524aa","question":"A large group of people are seen standing behind one another with others watching on the sides and leads into the group running down a road. the large group of kids\n\nA. then stand still and each takes turns running down a different road several times as the people watching continue to laugh and then come back down the road in the end.\nB. continue running down the road after one another while people still watch on the sides.\nC. then play hockey together while one another run all around and end by off scoring a goal and the camera panning around to people walking around and looking at the score.\nD. continues when many close up do short stunts on hole slides in the street while the camera captures them from several angles.","prompt":"A large group of people are seen standing behind one another with others watching on the sides and leads into the group running down a road. the large group of kids","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3549,"error":null,"output":"B"}
{"question_id":"hellaswag:00008428","item_index":40,"attempt":0,"prompt_hash":"436e176808bc2a2542d4b5a1b5fc6af4542ce4c473a9931b9b8e83b9445454b5","question":"[header] How to help a friend with a broken heart [title] Encourage grieving. [step] Your friend needs to deal with their emotions in order to get through this tough time, so encourage them to tackle them head-on. Remind them that they will never feel better if they are in denial about what has happened or ignores how they are feeling about it.\n\nA. This can give them a chance to recognize they cannot control the reality of what they have lost and how they feel about it. [substeps] For example, if your friend has a broken leg, you may offer to join them in healing to help them move through the grief process.\nB. [substeps] Set up a time to talk about what you are going through. You might say, \" do you have a minute? There is something we need to tell you, and this is a great time to talk about it.\nC. [substeps] Let them know it's okay to cry. Tears can help them heal! If you feel like your friend is bottling their emotions up inside, explain to them that doing so can make it harder to get past the hurt.\nD. Try to debrief the friend while they are trying to cope with their emotional trauma. [substeps] If you are in charge of monitoring the person, it's more difficult to arrive at solutions as they are in a vulnerable moment.","prompt":"[header] How to help a friend with a broken heart [title] Encourage grieving. [step] Your friend needs to deal with their emotions in order to get through this tough time, so encourage them to tackle them head-on. Remind them that they will never feel better if they are in denial about what has happened or ignores how they are feeling about it.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3544,"error":null,"output":"C"}
{"question_id":"hellaswag:00002465","item_index":41,"attempt":0,"prompt_hash":"e7b9d9d37897f2c7588ee38ed5f0cd12927fab44f4f2496105114f9cc3fc0e66","question":"A person is see handing an officer an ice cream cone while a woman watches on the side. The woman then speaks to the officers while others watch on the side. the man\n\nA. continues to get the cone and ice cream and a little girl puts hers in her mouth.\nB. then grabs an ice cream and hands it to the woman.\nC. then resumes shoveling the ice cream cone and the woman and the man talk.\nD. then picks up a the ice cream scoop and licks off the bottom.","prompt":"A person is see handing an officer an ice cream cone while a woman watches on the side. The woman then speaks to the officers while others watch on the side. the man","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3546,"error":null,"output":"B"}
{"question_id":"hellaswag:00006143","item_index":42,"attempt":0,"prompt_hash":"f6a51c8d0cdcd0fdfa000a017526474c1b8494397a89073b1eab760da694e47f","question":"[header] How to keep a wine journal [title] Consider a regular notebook for use as a wine journal. [step] The most economical option is to buy a standard spiral-bound ruled notebook. These notebooks are very inexpensive, and you can organize your tastings chronologically by simply recording each wine on the next free page.\n\nA. This will allow you to only write in first person notes if you wish to remember all of the details of each alcoholic. [substeps] Don't use pint glasses or other cups as this will literally separate wine from booze and give you a mouth-closing environment.\nB. You can also write down suggestions or potential-potential-potential-opportunity questions you might ask yourself while you're writing down notes. [substeps] For example, if you want to discover great wine recipes, it is a good idea to make a journal and keep it and notes ready before you actually make the wine.\nC. [title] Purchase a ringed binder for increased organization. [step] Another option is to purchase a ringed binder with a set of dividers.\nD. [substeps] Write down notes about the bottles and who got them, as well as their date and number in front of a blank page. That way, you can retrack the figures as needed.","prompt":"[header] How to keep a wine journal [title] Consider a regular notebook for use as a wine journal. [step] The most economical option is to buy a standard spiral-bound ruled notebook. These notebooks are very inexpensive, and you can organize your tastings chronologically by simply recording each wine on the next free page.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3546,"error":null,"output":"C"}
{"question_id":"hellaswag:00009955","item_index":43,"attempt":0,"prompt_hash":"a3e8d6d1501be7ded96d67cd74d7e38b20503d5d93d3a41cf3eb9239fe2e7236","question":"[header] How to decide whether or not to get your ears pierced [title] Consider your age. [step] There is no legal age restriction on when you can get your ears pierced, but you will need signed consent from a parent if you are under the age of sixteen. The average age that people get their ears pierced is seven, but it ranges from infant to adulthood.\n\nA. If you are under the age of eighteen, and have put in hours of work on your ears, you are legally at the age when you could get your ears pierced. [substeps] Your peers are required to consent as well.\nB. [substeps] If you are in school it is important to consider your school's rules on body piercings. If they are not allowed, you might want to wait to get them pierced.\nC. [substeps] While your hearing just increases with age, you will need to have pierced your ear as a teenager to eventually want your ears pierced again. Because of this, you will probably need permission first before you can get your ears pierced permanently.\nD. Anyone over the age of seven could get pierced, except anyone older than six months old. Do not be afraid to ask your parents or guardians for the consent.","prompt":"[header] How to decide whether or not to get your ears pierced [title] Consider your age. [step] There is no legal age restriction on when you can get your ears pierced, but you will need signed consent from a parent if you are under the age of sixteen. The average age that people get their ears pierced is seven, but it ranges from infant to adulthood.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3544,"error":null,"output":"B"}
{"question_id":"hellaswag:00008595","item_index":44,"attempt":0,"prompt_hash":"dcc312b81eac1399e013fd402e2a33581e7d00dbafc194f4aca574f84011a9f1","question":"[header] How to choose nail polish colour that suits you [title] Go for nude shades if you have a light or medium skin tone. [step] Nudes tend to work best for light-coloured or medium-coloured skin. Lighter skins benefit from a pinkish nude.\n\nA. [title] Create your own colour to go with the nail polish color. [step] If you are less tanned, don't wear nail polish that is harder to peel apart or to glue down by using a popsicle stick to prime the base portion.\nB. Avoid beige if you have pale skin, as it could create a yellow undertone. If you have medium-coloured skin, you can go for either beige nudes or nudes with a pinkish undertone.\nC. [substeps] This style brings out a rosy pink finish. It brightens skin for small people, but for people who don't mind more vibrant shades, a nude colour keeps the skin supple and natural.\nD. [title] Pick nude nail polish for applying if you have medium skin tones. [step] They can help smooth your skin but have uneven edges that you might be able to see through.","prompt":"[header] How to choose nail polish colour that suits you [title] Go for nude shades if you have a light or medium skin tone. [step] Nudes tend to work best for light-coloured or medium-coloured skin. Lighter skins benefit from a pinkish nude.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3540,"error":null,"output":"B"}
{"question_id":"hellaswag:00005132","item_index":45,"attempt":0,"prompt_hash":"295e44b50ec377e0f8c6a250cab4a4e994e944e15276c11d684111be5f04d7e7","question":"[header] How to apply a spray underarm deodorant [title] Remove the appropriate pieces of clothing. [step] If you are applying deodorant without taking a shower first, take off your shirt and anything you may be wearing underneath that. Remove your clothes so you don't spray them along with your skin.\n\nA. You may want to wipe a spare towel or a washcloth under your underarm for extra protection. [substeps] Make sure that you remove any jewelry/accessories that you may be wearing underneath your shirt for a very non-avoidable effect.\nB. This avoids the spread of the insect repellent on your skin. [substeps] If you use a perfume or antiperspirant, put it on to stop the spread of the insect repellent.\nC. [title] Gather your supplies for the underarm spray. [step] Use a large, clean t-shirt or towel to apply deodorant to your underarm.\nD. Keep them clean so you can tell later on if you are using too much, if and when traces of deodorant appear on your clothes after contact with your skin. [substeps] Deodorant can also be applied to your feet to reduce their smell and the risk of blisters.","prompt":"[header] How to apply a spray underarm deodorant [title] Remove the appropriate pieces of clothing. [step] If you are applying deodorant without taking a shower first, take off your shirt and anything you may be wearing underneath that. Remove your clothes so you don't spray them along with your skin.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3539,"error":null,"output":"C"}
{"question_id":"hellaswag:00009034","item_index":46,"attempt":0,"prompt_hash":"f712c3593276bb469e5b1e0eb60a4fc8bd36df69796d7786e8283f8fa22a9199","question":"[header] How to set up a fish tank (for goldfish ) [title] Consider the size of your fish tank. [step] Goldfish require particularly roomy living arrangements in order to stay healthy. While they are pretty small fish, they need larger tanks than you might expect.\n\nA. [substeps] You can do better than a bowl. Despite the beauty of a goldfish suspended in a sphere of glass, most fishbowls simply don't provide enough room for their occupants.\nB. The fish will require more room to move around during the day than they would in an aquarium. [title] Choose a small tank for the goldfish to retreat into.\nC. Goldfish can only live in one tank, so it's best if you have enough extra fish or larger tank plants to accommodate a wide variety of fish. [substeps] Remember that goldfish are not omnivores.\nD. For example, if you have a 20 gallon tank and 4 fish, you'll likely need 20 gallons. [substeps] If you have a larger tank, consider a 3 foot (90 cm) tank.","prompt":"[header] How to set up a fish tank (for goldfish ) [title] Consider the size of your fish tank. [step] Goldfish require particularly roomy living arrangements in order to stay healthy. While they are pretty small fish, they need larger tanks than you might expect.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3540,"error":null,"output":"A"}
{"question_id":"hellaswag:00008776","item_index":47,"attempt":0,"prompt_hash":"6891f699b6f5b1482416227564381106de2643ee465418c54221ccc7080dd138","question":"[header] How to present yourself and business powerfully [title] Show your confidence in yourself. [step] When you make a presentation, whether to one person when you attend networking events or to 100 people when you make a formal presentation, project competence and authority, regardless of your position, to move ahead in your career or move your business to the next level. First impressions are crucial.\n\nA. Try to address people, not just yourself, through your act of speaking. [substeps] Be nice when approaching each person, including your boss, co-workers, or business associates.\nB. Use good manners and greeting to point people toward your own strengths. [substeps] Avoid impolite and unprofessional comments, which can come across as wrong.\nC. Make sure your statements are concise, not rhetorical. This shows you respect the values of others and is an asset for you as a business person.\nD. Often, the first impression you make is when you shake someone's hand in greeting. Your handshake communicates critical information.","prompt":"[header] How to present yourself and business powerfully [title] Show your confidence in yourself. [step] When you make a presentation, whether to one person when you attend networking events or to 100 people when you make a formal presentation, project competence and authority, regardless of your position, to move ahead in your career or move your business to the next level. First impressions are crucial.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3542,"error":null,"output":"D"}
{"question_id":"hellaswag:00004726","item_index":48,"attempt":0,"prompt_hash":"b10b99194dce07f003ef6a9674b33e60e08c06ff8411e5cc0b101626f8b9a882","question":"[header] How to shop for a wedding dress [title] Decide when to shop. [step] There's no time too early to start looking, but there is a time that's too late. Depending on where you get your dress, it could take several months for your dress to be ordered and shipped.\n\nA. From then, you'll need several weeks for your seamstress to make alterations. You should start shopping about 9 months before your wedding, but preferably sooner.\nB. If you aren't sure about the exact times, make a list. [substeps] If you're unable to visit the store and the dress is not shipped out in advance, sit down and plan a shopping trip around that time.\nC. If you get your dress shipped at the last minute, that's even worse. Some wedding dress designers are willing to let you pick the bride/groom's dress, so don't make this a trial run.\nD. One such example would be march 1st, and the exact timing might vary slightly by weather. In situations where you normally buy on cold winter days, however, it may be best to shop before your wedding day or if possible if it's crowded.","prompt":"[header] How to shop for a wedding dress [title] Decide when to shop. [step] There's no time too early to start looking, but there is a time that's too late. Depending on where you get your dress, it could take several months for your dress to be ordered and shipped.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3764,"error":null,"output":"A"}
{"question_id":"hellaswag:00009526","item_index":49,"attempt":0,"prompt_hash":"7f803802293968a7d342cd137163409f352f27fe11c373f157d57ee6e2fc7aa6","question":"[header] How to measure your waist to height ratio [title] Get the right supplies ready. [step] You need a few things in order to calculate your waist-to-height ratio. Having everything ready will make this process quick.\n\nA. [substeps] The first thing you'll need is a tape measure. Get a non-elastic, cloth measuring tape.\nB. Luckily, many calculators are available online at many major retailers. Have enough goody bags and a measuring tape to accommodate everything you need.\nC. You'll need a tape measure, ruler, measurement calculator, order printed paper, binder clips, measuring tape and a ruler with a round tip. [substeps] Always measure from your mid-chest up to your waist.\nD. If you don't have supplies, you could just buy some unused ones from a grocery store. You'll need : [substeps] A ruler or tape measure ruler sewing machine thread scissors paper clip sewing wire cardboard baking paper scissors pins pressure cooker (help ease tension) measuring tape measuring tape plastic or wooden clothespins.","prompt":"[header] How to measure your waist to height ratio [title] Get the right supplies ready. [step] You need a few things in order to calculate your waist-to-height ratio. Having everything ready will make this process quick.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3539,"error":null,"output":"A"}
{"question_id":"hellaswag:00006051","item_index":50,"attempt":0,"prompt_hash":"06957cb9e0faf5e04b2190084732000bce7ee61a2cded64b0fd5e6eb2309b48b","question":"[header] How to unclog a drain with salt and vinegar [title] Mix salt and vinegar. [step] In a small bowl, pour 1 cup of salt. Add 1 cup of vinegar.\n\nA. Place 1 cup of distilled white vinegar in a small bowl. The salt and vinegar will help the solution penetrate the clogged drain.\nB. Stir well so the salt soaks up all the vinegar. Continue stirring until the mixture is smooth and even.\nC. Mix the salt in, going slowly but bringing the mixture to a boil. Always stir the solution well, making sure nothing gets broken.\nD. If possible, give it an extra stir. The vinegar will set the clog, and it will give it a smoother initial scrubbing.","prompt":"[header] How to unclog a drain with salt and vinegar [title] Mix salt and vinegar. [step] In a small bowl, pour 1 cup of salt. Add 1 cup of vinegar.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3540,"error":null,"output":"B"}
{"question_id":"hellaswag:00002530","item_index":51,"attempt":0,"prompt_hash":"d220ca1842027b6632befb6992341a4057ce4cd222de68a40409780b47351ab9","question":"A large group of people are seen standing around a gymnasium looking at one another. One man is seen standing on a ladder and speaking to the group. the people\n\nA. continue to work out with one another and end by jumping off the ladder and clapping at the end.\nB. move around slowly and leads into them walking around with a tray in their hands.\nC. then begin bungee jumping off a small ramp into the pool and the men are seen smiling to the camera.\nD. then begin playing a game of volleyball.","prompt":"A large group of people are seen standing around a gymnasium looking at one another. One man is seen standing on a ladder and speaking to the group. the people","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3539,"error":null,"output":"D"}
{"question_id":"hellaswag:00000912","item_index":52,"attempt":0,"prompt_hash":"31d7438ff4aa9a48f58bf8248a19ee96b21add0bea33f6840099c49b9a89af0a","question":"A person sits in an inter tube at the top of a hill then rides down the slope laughing. The family straps together the inter tubes in a circular grouping with the kids sitting in them. the family\n\nA. then rides together floating through a tunnel and getting stuck on the tubes but eventually out themselves.\nB. squeezes through the tubes in the tunnel and then continue riding down the hill.\nC. rides in a large grouping of inter tubes down a snowy slope.\nD. concentrates on tight intertubes and the bridge above their heads.","prompt":"A person sits in an inter tube at the top of a hill then rides down the slope laughing. The family straps together the inter tubes in a circular grouping with the kids sitting in them. the family","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3763,"error":null,"output":"C"}
{"question_id":"hellaswag:00002717","item_index":53,"attempt":0,"prompt_hash":"6bec316fbfafc1f14fee8afc91d907fdf6336a279026e2b2ddefc08ba8036ab2","question":"A man is standing on ice holding a fishing pole in his right hand and there's a very small hole in the ice in front of him and he drops his hook into the hole and tugs on his pole. the man\n\nA. fishes intensely and retrieves the hook melt while still pulling on his stick, till his fish kiss from the hole in the ice.\nB. pulls a fish out of the holes on a string from his behind and an ice fishing pole goes into the hole.\nC. pulls a fish from his hook that was stuck in the ice and a fish resembling a shark rush out to reach it and grab the bug and quickly turn its from view.\nD. then pulls on the fishing wire a few times and stands there holding his fishing pole, moving it, and winding it from time to time.","prompt":"A man is standing on ice holding a fishing pole in his right hand and there's a very small hole in the ice in front of him and he drops his hook into the hole and tugs on his pole. the man","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3539,"error":null,"output":"B"}
{"question_id":"hellaswag:00007551","item_index":54,"attempt":0,"prompt_hash":"625fd6bbdc5bdeee9806e716fc2bdcede5ca878ce5a2a54b9a655a6505d0c6e9","question":"[header] How to fundraise for your youth sports team [title] Figure out what you are fundraising for. [step] There may be several reasons why your sports team needs to fundraise, but be specific. Create a detailed list of all the things your team needs money for.\n\nA. This could be the things that support campus events or school functions. Make sure to talk to your office manager about this list.\nB. [substeps] You can easily deduct what you need from the fundraiser. Each year, add in a monetary donation such as money for a sponsor or a raw food donation from your neighborhood.\nC. Here are some common examples of what sports teams need money for : [substeps] New equipment travel cost team uniforms tournament fees program operations facility rental or repair new stadiums or fields athletic scholarships [title] Set your fundraising goal. [step] After you've created a list of your team's needs, set a funding goal.\nD. If your team is not at war with the united states military, you may need to hire an organization to run it. If you are running the company with no equipment, this may be extra difficult.","prompt":"[header] How to fundraise for your youth sports team [title] Figure out what you are fundraising for. [step] There may be several reasons why your sports team needs to fundraise, but be specific. Create a detailed list of all the things your team needs money for.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3538,"error":null,"output":"C"}
{"question_id":"hellaswag:00009603","item_index":55,"attempt":0,"prompt_hash":"cd2980f9d0578a53f9f7239d1e60c785ea141bc91113bef4acacbe6464a462ac","question":"[header] How to oil trumpet valves [title] Choose a valve oil. [step] Valve oil helps clean and lubricate the trumpet, which will help you play your trumpet more efficiently, and can be purchased at most music stores or at various online retailers. There are two types of oils that are recommended to use for brass instruments: synthetic-based oils and petroleum-based oils.\n\nA. [title] Obtain a brass or other hardcase tumbler to store your brass or other materials. [step] Brass needs to have a lid, and the tumbler must be something made of tempered plastic or a pressure plate.\nB. [substeps] Synthetic oils are commonly found in brass instruments and are much like petroleum oil except they contain less oxidation. Petroleum based oils are not used to clean and lubricate brass instruments.\nC. [substeps] Synthetic oils use unregulated, high-efficiency lubricants and are not approved by the fda to provide lubrication for brass instruments. Petroleum oil can be purchased at most smaller, household instruments stores or online.\nD. Deciding between the two is often a matter of personal preference and determining which works better with your instrument. [substeps] Petroleum-based oil works best with brand new trumpets.","prompt":"[header] How to oil trumpet valves [title] Choose a valve oil. [step] Valve oil helps clean and lubricate the trumpet, which will help you play your trumpet more efficiently, and can be purchased at most music stores or at various online retailers. There are two types of oils that are recommended to use for brass instruments: synthetic-based oils and petroleum-based oils.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3538,"error":null,"output":"D"}
{"question_id":"hellaswag:00009692","item_index":56,"attempt":0,"prompt_hash":"a4b8990844500fdf9a14d4fd9b40c8c3d479edd107ab1f8f6b88f816000d0500","question":"[header] How to paint ceiling corners [title] Remove any furniture that you can from the room. [step] Trying to paint ceiling corners with a bunch of furniture in the way will only make a hard job even harder. Plus, you risk dripping paint on your furniture by accident.\n\nA. Loose furniture will also make going in and out of the room more difficult. It's best to just get out of the room and let the walls dry.\nB. Remove as much furniture as possible from the room and make sure it is out of the way before you paint it. [substeps] Be sure not to soak up any paint with your bare hands.\nC. Pull furniture away from the walls and move it to the center of the room before you get started. [substeps] If you're painting the whole ceiling and not just the ceiling corners, remove your furniture to another room entirely.\nD. [title] Lay down furniture in a circular pattern on the floor. [step] If you have a lot of furniture in the room, it's a good idea to put it all on a flat surface so that you can get a good idea of where each spot should go.","prompt":"[header] How to paint ceiling corners [title] Remove any furniture that you can from the room. [step] Trying to paint ceiling corners with a bunch of furniture in the way will only make a hard job even harder. Plus, you risk dripping paint on your furniture by accident.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3760,"error":null,"output":"C"}
{"question_id":"hellaswag:00009336","item_index":57,"attempt":0,"prompt_hash":"16884912b62fa09099f55d57c59645a2dd413169f7913c71cbb37397d57bfa3f","question":"[header] How to do subtle yet dramatic makeup [title] Apply a white eyeshadow to the inner corner of your eyes. [step] Apply it on your eyebrow bone too. This will highlight your eyes, and make them appear bigger and brighter.\n\nA. [title] Take a medium grey eyeshadow and gently spread it over the rest of your eye. [step] Take it slightly past the crease and edge.\nB. Use a store-brand eyeshadow. [title] Trace your eyelid using a black liner to create the crease.\nC. [title] Create some lace on each eye, going from above your eyebrow up to the middle. [step] From the outer corner of the eye outward, make a loop in the lace.\nD. [title] Apply yellow, orange, or purple eyeliner to your upper lids. [step] Thinly coat the top lid with yellow eyeliner.","prompt":"[header] How to do subtle yet dramatic makeup [title] Apply a white eyeshadow to the inner corner of your eyes. [step] Apply it on your eyebrow bone too. This will highlight your eyes, and make them appear bigger and brighter.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3760,"error":null,"output":"D"}
{"question_id":"hellaswag:00001410","item_index":58,"attempt":0,"prompt_hash":"2d70ea24eba35ce8f844fd57eded1f99cea92b26e660f40df248aff40fc00075","question":"A group of people are skiing down the side of a mountain. A group of moose are baying between scenes of lifts and helicopters overhead. people\n\nA. are snowboarding down the slide.\nB. are skiing through snowy tunnels and cliffs.\nC. climb up the side of the mountain and snowmobile down the mountain.\nD. give each other high fives as they ski.","prompt":"A group of people are skiing down the side of a mountain. A group of moose are baying between scenes of lifts and helicopters overhead. people","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3338,"error":null,"output":"D"}
{"question_id":"hellaswag:00008246","item_index":59,"attempt":0,"prompt_hash":"7df88b2d3f63de3912357bcce07d134e4b97b500ac9434de6667ed8af8b01f56","question":"[header] How to archive a slack channel on android [title] Open the slack app on your android. [step] The slack app looks like an \" s \" in a colored circle icon on your apps menu. [substeps] If you're not automatically logged in, tap the blue sign in button in the lower-left corner, and sign in to the workspace you want to edit.\n\nA. [title] Tap and hold the slack icon on your contacts. [step] This icon will open your contacts list and show you several apps.\nB. [title] Tap a scene you'd like to archive. [step] It's in the menu at the top of the screen, to the left of the \" channels \" option.\nC. [title] Tap the three horizontal lines icon on the lower-left corner. [step] It's near the top of the screen.\nD. [title] Tap the workspace icon on the top-left. [step] This button looks like the initials of your workspace's name in the upper-left corner of your screen.","prompt":"[header] How to archive a slack channel on android [title] Open the slack app on your android. [step] The slack app looks like an \" s \" in a colored circle icon on your apps menu. [substeps] If you're not automatically logged in, tap the blue sign in button in the lower-left corner, and sign in to the workspace you want to edit.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3896,"error":null,"output":"D"}
{"question_id":"hellaswag:00002413","item_index":60,"attempt":0,"prompt_hash":"3d377965f45fd9f68d34a384613944cbb69ec50440c016c846f0c1a88d54b593","question":"A close up of a violin is seen with a person's hands moving all around. the man\n\nA. closes a book and shows a tune next to the instrument.\nB. uses a brush to rub sumo art all over the neck of the violin followed by pushing it back.\nC. begins playing the violin while moving his hands up and down.\nD. holds the violin up and continues playing while moving his hands all around.","prompt":"A close up of a violin is seen with a person's hands moving all around. the man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3895,"error":null,"output":"C"}
{"question_id":"hellaswag:00007067","item_index":61,"attempt":0,"prompt_hash":"7d7d1a2f348641297905333766f5bdd0a91c4e949555de7239f1329129e87d33","question":"[header] How to bond with a stray cat [title] Safety first. [step] Don't compromise your own safety. This includes not going into any area that may be dangerous, such as a construction site, an abandoned mining area or a dilapidated building.\n\nA. Moreover, don't put yourself in a precarious position to try to reach the cat. Make sure you are on a solid surface that won't fall out from under you or cause you to slip or fall.\nB. Instead, gather supplies and tools you won't likely need as you get a stray cat. [substeps] Grab paints, gauze (or a few strips of black gauze), roughly 2 tablespoons (15 ml) of fabric, or masking tape.\nC. You might also be insulted if your stray cat bumps into a neighbor or another old friend. Stray cats will not behave well if people who encounter them leave them alone.\nD. Safety first means safety without being outside in tightly enclosed spaces. If the stray cat is new to the home, or seems scared or panicked, or continues to keep circling its territory, there is a good chance it's being tracked.","prompt":"[header] How to bond with a stray cat [title] Safety first. [step] Don't compromise your own safety. This includes not going into any area that may be dangerous, such as a construction site, an abandoned mining area or a dilapidated building.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3895,"error":null,"output":"A"}
{"question_id":"hellaswag:00000685","item_index":62,"attempt":0,"prompt_hash":"823b72bd8a166183352c4079fdbf3d4cceca6d0bc7ac8c9f0adc4845fbba1ade","question":"The person kneels next to his car and shows cleaning products. The man shakes up the product before applying it to a sponge. the man\n\nA. waxes down the door of the car with the product.\nB. rubs the sponge in foam on the cars tires.\nC. rubs the wet area of the windshield then wipes it off with a sponge to remove the germs.\nD. puts it under the tire of the car.","prompt":"The person kneels next to his car and shows cleaning products. The man shakes up the product before applying it to a sponge. the man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3895,"error":null,"output":"A"}
{"question_id":"hellaswag:00000168","item_index":63,"attempt":0,"prompt_hash":"be947573d8171695df34d63e34a6999ed9b53b74ed5b5e0f05af5bc06b7f4eb9","question":"There is a woman eating ice cream from a tub and talking to the camera man. And he tries to look into the tub, but she pulls it away. the camera person\n\nA. continues talking to the social media to find a rich husband in that world.\nB. is sitting in the bushes watching the woman.\nC. sits down to continue talking to the woman.\nD. stops at a jar of liquid in the sink.","prompt":"There is a woman eating ice cream from a tub and talking to the camera man. And he tries to look into the tub, but she pulls it away. the camera person","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3895,"error":null,"output":"C"}
{"question_id":"hellaswag:00002229","item_index":64,"attempt":0,"prompt_hash":"5cddd0928cc781cef3a062ba19262158bfafb7786ae0a1c3a066a586e0b98a4d","question":"The man drills the cabinets apart and wipes and paints them. The cabinets dry and the man wipes them. we\n\nA. see a boy in a petting zoo covered with paint and foam animals.\nB. see a man talking through a camera.\nC. return to interviewing and sanding the wall.\nD. see a person applying a product over the paint.","prompt":"The man drills the cabinets apart and wipes and paints them. The cabinets dry and the man wipes them. we","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5083,"error":null,"output":"D"}
{"question_id":"hellaswag:00002095","item_index":65,"attempt":0,"prompt_hash":"54481fc7f3814eede0f87af0de6ec2fa3fd9e3272eba4ad9f06635e100bcd154","question":"He gently tries to pull up and then reel the fishing line out of the hole. He dips and lowers while taking a break. He tries to reel the line out again. he\n\nA. looks up from his shallow hole rubbing his face.\nB. pulls up, takes a few drags and starts reeling again.\nC. removes his gloves and gets to his knees and back up as he tries to pull the line out of the hole repeatedly.\nD. reels the fish out once more and pulls it out of the hole.","prompt":"He gently tries to pull up and then reel the fishing line out of the hole. He dips and lowers while taking a break. He tries to reel the line out again. he","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5082,"error":null,"output":"D"}
{"question_id":"hellaswag:00003682","item_index":66,"attempt":0,"prompt_hash":"d2f31fd470c55d1f72cba0264b0aaebaa09c8fa15506203f7db5ff2d1570c1d0","question":"[header] How to modify a prenuptial agreement [title] Go over the agreement with your spouse. [step] Review each provision in your prenuptial agreement to decide which provisions you and your spouse want to modify. [substeps] A prenuptial or postnuptial agreement is only valid after full disclosure from both you and your spouse regarding your income, assets, debts, and liabilities.\n\nA. If any of this information has changed since you signed the prenup, share it openly. Don't assume that your spouse knows.\nB. [title] Read and review the prenuptial agreement. [step] Include any clauses or provisions which you or your spouse agree to modify.\nC. [title] Obtain a copy of the prenuptial agreement. [step] Once you've reviewed all of the provisions in the agreement, you'll want to make sure you have an exact copy of the prenuptial agreement.\nD. With postnuptial agreements, you are legally bound by the terms you satisfy with the other spouse. [title] Draft the proposed or modified prenuptial agreement.","prompt":"[header] How to modify a prenuptial agreement [title] Go over the agreement with your spouse. [step] Review each provision in your prenuptial agreement to decide which provisions you and your spouse want to modify. [substeps] A prenuptial or postnuptial agreement is only valid after full disclosure from both you and your spouse regarding your income, assets, debts, and liabilities.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5076,"error":null,"output":"C"}
{"question_id":"hellaswag:00002231","item_index":67,"attempt":0,"prompt_hash":"492c6574383d53ef1f54d6346f39bc853f3d041edceca84cc831e3355ab5f8dd","question":"We see a person applying a product over the paint. We see the cabinet faces put back on the cabinets. we\n\nA. see the finished whiskey bottle.\nB. see the paint laid out on a counter.\nC. see the end credits on a black screen.\nD. see the finished glassware.","prompt":"We see a person applying a product over the paint. We see the cabinet faces put back on the cabinets. we","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5079,"error":null,"output":"C"}
{"question_id":"hellaswag:00006529","item_index":68,"attempt":0,"prompt_hash":"b935edd0958eee5fcc2eda570ef5a14ccce848149bc142e20ee32f6d4aa6e177","question":"[header] How to be safe during a heat emergency [title] Stay indoors as much as possible and limit exposure to the sun. [title] Stay on the lowest floor out of the sunshine if air conditioning is not available. [title] Consider spending the warmest part of the day in public buildings such as libraries, schools, movie theaters, shopping malls, and other community facilities.\n\nA. [title] Keep at least 16 people in each room at all times. [title] Try to keep your doors and windows closed when rooms are steamy or fans are blowing.\nB. [substeps] Being further away from these structures will only end up making the heat worse. [title] Explain the essential ahead of time to friends and family that you will be having a fire to light up the building.\nC. [step] Circulating air can cool the body by increasing the perspiration rate of evaporation. [title] Eat well-balanced, light, and regular meals.\nD. [title] When the weather is pleasant, use umbrellas and any coverings you can find at stores. [title] Expose your hands, torso, and legs to the sun.","prompt":"[header] How to be safe during a heat emergency [title] Stay indoors as much as possible and limit exposure to the sun. [title] Stay on the lowest floor out of the sunshine if air conditioning is not available. [title] Consider spending the warmest part of the day in public buildings such as libraries, schools, movie theaters, shopping malls, and other community facilities.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5076,"error":null,"output":"C"}
{"question_id":"hellaswag:00004808","item_index":69,"attempt":0,"prompt_hash":"0f08c3b4f9cb6379a3522932da473392a4ec12e92306a72360ad11e92ed72639","question":"[header] How to choose and start wearing makeup [title] Try to figure out what kind of skin tone you have. [step] Look at the veins on your wrist. If they are a greenish tone you have a warm skin tone, if they are blue or purple, you have a cool skin tone.\n\nA. Get fitted for makeup by a makeup specialist and try different colour combinations to see which one appeals to you. [title] Consider running a stain test on your wrist.\nB. If they are red and are shaped exactly like grape veins you have a neutral skin tone. You may have the veins on your elbows.\nC. A cool skin tone is warm in color and lustrous brown with a purple undertone, giving it a cool appearance and looking flawless. [title] Look at the flecks of red in your eye after washing.\nD. If you have both, your skin could be neutral. Different kinds of colors go with different skin tones.","prompt":"[header] How to choose and start wearing makeup [title] Try to figure out what kind of skin tone you have. [step] Look at the veins on your wrist. If they are a greenish tone you have a warm skin tone, if they are blue or purple, you have a cool skin tone.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5077,"error":null,"output":"D"}
{"question_id":"hellaswag:00001908","item_index":70,"attempt":0,"prompt_hash":"99e2e8d0236c9c80ec5cff6f62ed14bf201f0a70e9e854f7e0f880b4f84feaba","question":"A man chugs an entire mug of beer. the friends\n\nA. watch the beers come.\nB. raises his arms and cheers.\nC. look away as the bartender drains it.\nD. shingles on a roof with him as some men ease the roof down and push it into place.","prompt":"A man chugs an entire mug of beer. the friends","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5078,"error":null,"output":"B"}
{"question_id":"hellaswag:00004873","item_index":71,"attempt":0,"prompt_hash":"de9fc3a1173aa72ffe5f946a84daa5b9741a86b77cfcc330545c9b03075b7f39","question":"[header] How to convince others to let you be vegetarian or vegan [title] Tell them why you do not want to eat meat. [step] Politely explain your reasoning, but be persistent. [title] Ask them why they are against not eating meat.\n\nA. [step] Explain how you've noticed that meat is bad for you and what you've tried to do to prevent feeding it to others. [title] Show them that being vegetarian is unhealthy.\nB. [step] This is a long and complicated conversation. [title] Make the points yourself, because you could be heard.\nC. [step] If it is against their religion or something else maybe you can meet halfway. Maybe you can be vegetarian once a week for a month and so on and so forth.\nD. [step] Always end the conversation on a positive note. [title] Explain that vegetarianism is all about moderation in the body.","prompt":"[header] How to convince others to let you be vegetarian or vegan [title] Tell them why you do not want to eat meat. [step] Politely explain your reasoning, but be persistent. [title] Ask them why they are against not eating meat.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5074,"error":null,"output":"D"}
{"question_id":"hellaswag:00004445","item_index":72,"attempt":0,"prompt_hash":"ca14456f0d50609a2d4bd464e138be8161ee3201d64a21f9bae3ff4df9092abc","question":"[header] How to look good at 40 [title] Change your face make-up to fit your age. [step] Our skin naturally changes over time. Imagine if you were using the make-up you used to cover those pimples in grade school! And even though you may not have noticed much, your skin changes from 25 to 30, from 30 to 35, and 35 to 40.\n\nA. It's high time to update your make-up-you want very thin coverage that makes you glow. It's your natural beauty that's ready to be center stage.\nB. How young do you look now? And what would you like someone to see? Wash, apply an eyeshadow, and shave. Keep the hairdo simple and don't wear very much.\nC. Every time you do this now it's to bring out your natural beautiful smile. [title] Wear style! [step] This essential for looking smart and sophisticated and basically any boy wants to be smart.\nD. According to pinterest, those with age 50's don't wear much makeup or anything at all. These are easily lost in reality, there are so many products online.","prompt":"[header] How to look good at 40 [title] Change your face make-up to fit your age. [step] Our skin naturally changes over time. Imagine if you were using the make-up you used to cover those pimples in grade school! And even though you may not have noticed much, your skin changes from 25 to 30, from 30 to 35, and 35 to 40.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5075,"error":null,"output":"A"}
{"question_id":"hellaswag:00007663","item_index":73,"attempt":0,"prompt_hash":"f1dd8727cd854a6ce3e4d80b8dc95e25c78a591460e5ac65a87cbe2bf9e5c9cf","question":"[header] How to speed up metabolism with fat calories [title] Eat enough calories. [step] You should consume enough calories for your body. Otherwise, your body will go into starvation mode, which means your metabolism will decrease dramatically to save energy.\n\nA. In starvation mode, you won't be burning many calories. To avoid this scenario, you should eat enough to match your resting metabolic rate.\nB. Eat to maintain this goal : [substeps] Take in about 2.5 calories per day. You can increase this number back up by around 2 to 5 percent (if you really need to) over a longer period of time (perhaps 8 or more days) to 150 or more calories weekly.\nC. Stay consistent with the amount of calories you eat each day to stay afloat. The world around you literally rests on your shoulders.\nD. Stick to a healthy weight of at least 150% fat, and consume approximately 5 , 200 calories daily. [substeps] Put on one or two pounds per week to give your metabolism time to catch up.","prompt":"[header] How to speed up metabolism with fat calories [title] Eat enough calories. [step] You should consume enough calories for your body. Otherwise, your body will go into starvation mode, which means your metabolism will decrease dramatically to save energy.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5073,"error":null,"output":"A"}
{"question_id":"hellaswag:00001015","item_index":74,"attempt":0,"prompt_hash":"64e80f2e7678bef053973412ae07c4ff3386b9bb3ddba25eddb0011fc5bc5fce","question":"He begins playing the harmonica. The man stops playing and dances in place. The man continues to play his harmonica. the man\n\nA. walks away from the audience.\nB. stops playing and writes on a napkin.\nC. finishes playing and lies down on a piece of grass.\nD. stop splaying and thanks the crowd as the closing credits appear.","prompt":"He begins playing the harmonica. The man stops playing and dances in place. The man continues to play his harmonica. the man","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5077,"error":null,"output":"B"}
{"question_id":"hellaswag:00001412","item_index":75,"attempt":0,"prompt_hash":"f9e80ab6d67e11d930fe09d7f9fb901e3c8a1ad532fb44021642e385da2c4719","question":". a carved pumpking with a light in it\n\nA. is being paints.\nB. glows on a counter.\nC. appears in a forest of trees.\nD. sits on a bed of ice.","prompt":". a carved pumpking with a light in it","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5076,"error":null,"output":"C"}
{"question_id":"hellaswag:00002202","item_index":76,"attempt":0,"prompt_hash":"e5ef373a0a5d296ac0e2ebb964434dd9c11bb35a262040db7b43f324ac06c1f6","question":"The man is telling the woman how to make another shot. She attempts the shot and makes it and they high five again. they\n\nA. sit down and play another round of beer pong.\nB. shake hands and the girl hugs her dad.\nC. continue talking and laughing.\nD. are back on the dancing floor dancing.","prompt":"The man is telling the woman how to make another shot. She attempts the shot and makes it and they high five again. they","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5076,"error":null,"output":"C"}
{"question_id":"hellaswag:00004524","item_index":77,"attempt":0,"prompt_hash":"aebaf2b01d518bc683ba31ffc5b4b230cede7897fe621c6151ee234a44a7a3ac","question":"[header] How to convert pierced earrings to clip on earrings [title] Purchase earring converters. [step] You can buy earring converters online or your local craft store. The converters are usually located in the jewelry making area.\n\nA. [substeps] If you're looking for a quick way to convert earrings into clip-on earrings, pick the watch number one and two. Select the watch number two, then select the watch number three.\nB. Check chain stores and online shops for earring converters. [substeps] If you aren't able to find them online, you can find them at sewing stores or online stores that carry jewelry supplies.\nC. Look for a converter that has a small tube/barrel on the back. [substeps] Most posts will fit into the barrel of the converter, but check the size of your posts just to be sure.\nD. You will need to take your earring size into account when selecting earring converters. [substeps] Consider always buying a size 3 earring converter.","prompt":"[header] How to convert pierced earrings to clip on earrings [title] Purchase earring converters. [step] You can buy earring converters online or your local craft store. The converters are usually located in the jewelry making area.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5072,"error":null,"output":"D"}
{"question_id":"hellaswag:00003074","item_index":78,"attempt":0,"prompt_hash":"72e6b7b03e1a58d9c7310e8ce29c2201b710e8474dc40bc437dd0d18f0ffb94c","question":"A field with many racing sail boats being rigged for a competition. a boy\n\nA. is shown seated with several close ups of dart board exhibits.\nB. stands with two playboys and hold a banner, tying a knot.\nC. rolls his sail boat to the water and begins sailing in a competition.\nD. rides his boat in a racing row boat doing jumps and tricks in pool decks during a race.","prompt":"A field with many racing sail boats being rigged for a competition. a boy","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5074,"error":null,"output":"C"}
{"question_id":"hellaswag:00002243","item_index":79,"attempt":0,"prompt_hash":"8241bb101722b21dc199c7100ef780f5f85ece45a10a020fb4b63c2c217618c1","question":"A man is seen speaking to the camera and leads into him holding a piece of paper and two objects. He puts the objects on a board and sharpens a knife along the object. he\n\nA. puts the pieces down on a fence and continues speaking to the camera.\nB. continues cutting around the object and shooting a bow and arrow to the target.\nC. continues speaking to the camera while sharpening the blade and pointing to the object.\nD. takes a drink and then holds up the knife.","prompt":"A man is seen speaking to the camera and leads into him holding a piece of paper and two objects. He puts the objects on a board and sharpens a knife along the object. he","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5075,"error":null,"output":"C"}
{"question_id":"hellaswag:00007224","item_index":80,"attempt":0,"prompt_hash":"58f403072b4763cb989270fa192f539874ebeff27ca261fceaeb9f4a4d030d0d","question":"[header] How to clean a vornado fan [title] Clean out dust in the outer grills using a vacuum cleaner brush. [step] You can use a brush attachment on a vacuum cleaner to clean out all of the dust from the outer grills. Make sure the fan is unplugged before vacuuming.\n\nA. [title] Place a cleaning rod in each section of the fan and squirt some cleaning fluid through each rod. [step] Avoid getting any grease in the vents or rest tray.\nB. Some of the cleaner should be removed from the outside if you use a cloth and brush as cleaning instructions are laid out in the manual. [substeps] Avoid cleaning the outer grills with dust because it can detach some of the insulation and contribute to the fan's cooling.\nC. Wipe down the inside of the cover every few minutes so you can put the fan back together. [substeps] For maximum efficiency, you should wash your vornado fan at least once a month, which is right after each dry run.\nD. Go slowly when vacuuming to ensure you clean out as much dust as possible. [substeps] This will only clean the outer part of the fan, not the inside blades.","prompt":"[header] How to clean a vornado fan [title] Clean out dust in the outer grills using a vacuum cleaner brush. [step] You can use a brush attachment on a vacuum cleaner to clean out all of the dust from the outer grills. Make sure the fan is unplugged before vacuuming.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5074,"error":null,"output":"D"}
{"question_id":"hellaswag:00006025","item_index":81,"attempt":0,"prompt_hash":"8b53755d3283f7c833db473dac1170dfbaa7ad8af209c8bdc18951b9bfbbca5a","question":"[header] How to create family unity [title] Use effective communication skills. [step] Each member of the family should feel important and valued. This includes you.\n\nA. Make sure you're in compliance with the requirements for your relationship. [substeps] Let each member have room to speak and be sensitive.\nB. Try not to engage in teasing, cursing, or emotional outbursts. You need to treat others as you would treat your own.\nC. In order for your family members to hear your needs, you have to express them. Be polite, but honest with your family during discussions.\nD. It is important that each member of the family stay informed regarding what is going on and who is still alive. The ability to socialize, communicate, and be an integral part of the family makes you significant.","prompt":"[header] How to create family unity [title] Use effective communication skills. [step] Each member of the family should feel important and valued. This includes you.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5075,"error":null,"output":"C"}
{"question_id":"hellaswag:00009411","item_index":82,"attempt":0,"prompt_hash":"40fa4437dc5d71915d9cbdbc3f8f97910d960faf07219335d7decac81ec797c8","question":"[header] How to make a simple leopard gecko enclosure [title] Place the tank on a sturdy surface. [step] The tank must be at least 10 gallons, but a 20 gallon would be better. One with sliding doors would make taming easier.\n\nA. And be sure to keep the temperature pretty low, as geckos tend to be more active during the summer months. [substeps] A tap or even a small bowl of water may do.\nB. [substeps] The tank must not become heavy in weight. [title] Cover the cage or box with plastic wrap.\nC. Use caution if putting on the floor as the under tank heater may burn it. [title] Put the uth under the tank on one side.\nD. It's also less exciting and expensive. [title] Cover the tank with a boogie board.","prompt":"[header] How to make a simple leopard gecko enclosure [title] Place the tank on a sturdy surface. [step] The tank must be at least 10 gallons, but a 20 gallon would be better. One with sliding doors would make taming easier.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5076,"error":null,"output":"C"}
{"question_id":"hellaswag:00009678","item_index":83,"attempt":0,"prompt_hash":"5f5fbbf925b873329743b81d5502efdd1fd204ff6528e1c996a3b0be20a3fed8","question":"[header] How to set up a sole proprietorship in ohio [title] Decide if you want a trade name. [step] As a sole proprietor, you can operate under your legal name. For example, jane t. smith can advertise her services using her name: \" jane t. smith, accountant.\n\nA. \" [title] Determine if your business is a sole proprietorship. [step] If your business is not a sole proprietorship, you can operate either alone or with either a partner.\nB. \" if you want to use a different name, you'll need to request a trade name. [substeps] A trade name is a great way to distinguish yourself in the market.\nC. \" the same can be said for any business registered on the property (including your home), such as a bank or business, store, supermarket, or other establishment. Each type of business is unique, so it's best to decide whether you want a business partnership, llc, or limited liability company.\nD. \" you can also opt for a less traditional name, if you wish. [substeps] If you prefer a traditional name, then you should first write down your legal name.","prompt":"[header] How to set up a sole proprietorship in ohio [title] Decide if you want a trade name. [step] As a sole proprietor, you can operate under your legal name. For example, jane t. smith can advertise her services using her name: \" jane t. smith, accountant.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5075,"error":null,"output":"B"}
{"question_id":"hellaswag:00002098","item_index":84,"attempt":0,"prompt_hash":"0a7c3336e40a7c50293072ce3bc7cc77b478cfe572f30e123313ae398370c8c2","question":"There are several spectators and judges watching them as they jump over the bar in a large indoor stadium. both the contestants\n\nA. jump in over the bar and are met by several other contestants who cheer for them and clap.\nB. run and each other over a large concrete structure at a super sunny location.\nC. are playing a game of pool while one makes a perfect jump and scores a point.\nD. successfully jump over the bar.","prompt":"There are several spectators and judges watching them as they jump over the bar in a large indoor stadium. both the contestants","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5075,"error":null,"output":"D"}
{"question_id":"hellaswag:00009609","item_index":85,"attempt":0,"prompt_hash":"ebdba2da7c40134920dfe9edb2002e41720bdb5c09d6e8c67884a05e63acfac0","question":"[header] How to apply shimmer powder on your face and body [title] Look for a powder with a very fine shimmer. [step] When you're picking out a shimmer powder, look for very fine, shiny powders with no visible flecks of glitter. A fine shimmer will ensure that you look glowing and dewy, as opposed to flashy and glittery.\n\nA. [substeps] Always test a powder powder before you use it. If it feels against your skin, it won't stay for long.\nB. Your shimmer brush for your look should come with a lightweight, natural shimmer brush so that it doesn't have the tone off other skin tones. [title] Choose a translucent powder to create a shimmery glow.\nC. [title] Choose a bronze or golden shimmer powder if your skin is dark or tan. [step] These colors will highlight your natural coloring and accentuate the warmth in your skin tone.\nD. If you use a powder with a very fine shimmer, you'll have a hard time achieving the ideal flawless look. [title] Select a light powder for skin tones.","prompt":"[header] How to apply shimmer powder on your face and body [title] Look for a powder with a very fine shimmer. [step] When you're picking out a shimmer powder, look for very fine, shiny powders with no visible flecks of glitter. A fine shimmer will ensure that you look glowing and dewy, as opposed to flashy and glittery.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5074,"error":null,"output":"C"}
{"question_id":"hellaswag:00003806","item_index":86,"attempt":0,"prompt_hash":"828480bfc183583119f97cf48405eba9aa43b0d24d5d81d4fb87a84342b1ff4c","question":"[header] How to dress when you are in your forties [title] Get your clothes tailored. [step] Most people can't afford to buy clothes made to their specifications. However, getting less expensive clothes tailored can make all the difference in how they look.\n\nA. [substeps] For example, go for a fitted, long-sleeved or sleeveless top that emphasizes your body. Depending on where you are in your thirties, you might also consider wearing pants that are fitted.\nB. If your clothes are form-fitting, you're going to look that much more pulled together. When in your 40s, being able to pull together a professional look is a must.\nC. [substeps] The general rule is to get your clothing tailored to fit every body type. The first thing to check out is how well you'd like to wear them, have the clothes tailored to fit each body type, and how well you'd like to use them in your life.\nD. [substeps] Professional tailor tailors are knowledgeable about tailored clothes for almost every gender. Ask colleagues, friends, or coworkers about the exact fit of an outfit they would suggest you get.","prompt":"[header] How to dress when you are in your forties [title] Get your clothes tailored. [step] Most people can't afford to buy clothes made to their specifications. However, getting less expensive clothes tailored can make all the difference in how they look.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5290,"error":null,"output":"B"}
{"question_id":"hellaswag:00008504","item_index":87,"attempt":0,"prompt_hash":"77a3a5540c555a74d17f5e863712561bcef44883f7a8c7317d81631ebff89b94","question":"[header] How to know what it means to love someone [title] Know that you focus on someone else's happiness, not just your own, when you are in love. [step] Being in love is about connecting with another person emotionally, caring for their happiness as if it was your own. When you love someone their fate becomes intertwined with yours.\n\nA. [title] Notice an accident that happened a long time ago. [step] This is usually part of the reason why we tend to overreact.\nB. You may feel sad when they cry, get angry when they are hurt, and celebrate with their successes. [substeps] If you can't wait to share good news with them or hear about their day, love is likely growing.\nC. [substeps] If someone you love is gone, they will more than likely not be able to love you back. And as a result, try not to allow your feelings to get the best of you.\nD. If you look at someone and realize that something about them makes them happy, you're in love. [substeps] Don't feel like you have to spend a fortune on someone just to like them.","prompt":"[header] How to know what it means to love someone [title] Know that you focus on someone else's happiness, not just your own, when you are in love. [step] Being in love is about connecting with another person emotionally, caring for their happiness as if it was your own. When you love someone their fate becomes intertwined with yours.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5291,"error":null,"output":"B"}
{"question_id":"hellaswag:00008762","item_index":88,"attempt":0,"prompt_hash":"92800c7121088c1671aea91759799887e638882ba5761d72ffd182a04c042809","question":"[header] How to make a homemade earring holder [title] Wrap a box lid with a colorful piece of wrapping paper. [step] Secure the paper under the lid with wrapping paper. [substeps] If the lid is small enough, you can use scrapbooking paper instead.\n\nA. [title] Print out your custom earring holder on cardboard. [step] Using a pencil or marker, trace your design onto the cardboard.\nB. [title] Add fabric to the box lid to create an oval shape for your earring holder. [step] Apply fabric glue along the outside perimeter of the lid to create the oval shape.\nC. [title] Cut a pair of white or black nylons 4 inches (10.16 centimeters) longer than your lid. [step] If your paper is a light color, use white nylons.\nD. [title] Make a handle out of bubble wrap. [step] Include any other decorative elements that you would like in the handle.","prompt":"[header] How to make a homemade earring holder [title] Wrap a box lid with a colorful piece of wrapping paper. [step] Secure the paper under the lid with wrapping paper. [substeps] If the lid is small enough, you can use scrapbooking paper instead.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5291,"error":null,"output":"A"}
{"question_id":"hellaswag:00009693","item_index":89,"attempt":0,"prompt_hash":"b607564018669809ceb06c23a1e2f32a1988081d2002f624044fa238802c058a","question":"[header] How to become a professional cyclist [title] Cycle every day. [step] Training and experience are the foundations of any sport, and for a professional cyclist, this means making lots of miles on the road. On average, you should train at least two hours a day for six days a week if you want to become a professional cyclist.\n\nA. [substeps] To get into your training program, skip the internet and start by navigating to the model of bike you would like to become into. Once you've learned how to cycle, start logging into your part 1 computer and working your way to being an expert cyclist.\nB. Exercising can increase your flexibility as well as improve your stamina over time. Try the following shortcuts : [substeps] Bike train.\nC. [substeps] Go for at least two hours a day for about 60 minutes three times a week. You should not cut out at least eight hours if you want to get a bike bike.\nD. If the weather is too cold to cycle outside, train on a stationary bike in a gym or at home. [substeps] Training for pro cycling usually takes up about 4-6 hours a day, though sometimes that might include time spent at the gym doing weights and building strength.","prompt":"[header] How to become a professional cyclist [title] Cycle every day. [step] Training and experience are the foundations of any sport, and for a professional cyclist, this means making lots of miles on the road. On average, you should train at least two hours a day for six days a week if you want to become a professional cyclist.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5291,"error":null,"output":"D"}
{"question_id":"hellaswag:00009951","item_index":90,"attempt":0,"prompt_hash":"df12de8fe448a04db2a99399797535111c19d9d0c411d35bcbeca3cf0ecf5901","question":"[header] How to be good with young children [title] Listen carefully. [step] If you are looking to bond with a child, start by paying attention when they speak. Kids are just finding their voices, and what they are saying is very important to them.\n\nA. Put everything they said down on paper and make time to stop in for a moment and listen. Later on, you'll need to fill that space with topics about younger children.\nB. As they get older they can learn to listen more than act. That doesn't mean you have to wait for a few hours to listen.\nC. To help them hear you, try to step back and observe things about the child's voice as they talk. Pay attention to their body language: they may make an abrupt, jerky movement when talking.\nD. Actively listen when a young child is speaking to you. [substeps] One way to demonstrate that you are listening is to make eye contact.","prompt":"[header] How to be good with young children [title] Listen carefully. [step] If you are looking to bond with a child, start by paying attention when they speak. Kids are just finding their voices, and what they are saying is very important to them.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5291,"error":null,"output":"D"}
{"question_id":"hellaswag:00009917","item_index":91,"attempt":0,"prompt_hash":"e1ed847d040a593d26c1b06aeae61f79465d045285e7a0943766c45d54c8a625","question":"[header] How to create a fairy garden [title] Select a container. [step] Fairy gardens are usually housed in small containers. Think about how much space you have.\n\nA. [substeps] You can place each fairy garden in a bowl with water at one end or in a container full of larger objects on either side. Fairy garden containers come with multiple sides to hold various fruits, vegetables, herbs, spices, trees, and flowers.\nB. The size of a three-gallon bucket will be best for the fairy garden. The container you choose should also accommodate a few larger fruits and vegetables.\nC. You may want to place one container in each room and the other in the room itself, so that the fairies can see each other. Think about how much room you need to accommodate the other fairies.\nD. If you're keeping your garden indoors, you'll need a smaller container. For an outdoor garden, you can use something larger.","prompt":"[header] How to create a fairy garden [title] Select a container. [step] Fairy gardens are usually housed in small containers. Think about how much space you have.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5684,"error":null,"output":"D"}
{"question_id":"hellaswag:00009975","item_index":92,"attempt":0,"prompt_hash":"4956271fc06d59c5bc373b3d2c312f05678391149e9827c470df1f6db1b62f4d","question":"[header] How to clean merrell shoes [title] Brush off small amounts of dirt regularly. [step] As merrell shoes are often used for outdoor activities, they're prone to building up dirt and other debris. Make a habit of cleaning merrell shoes after each use.\n\nA. After wearing your shoes, use a soft brush to gently brush off any dirt or debris. [substeps] If you're vigilant about brushing off dirt regularly, it's less likely to build up on your shoes.\nB. Use a soft bristle brush to wet the surface of the shoe. Use a smooth cloth to rub away the dirt and many large clumps of mud.\nC. Every time you boot up the boat, take several moments to gently brush dirt off the shoes. Once you've finished this scrub, let the shoes air dry while you get dressed.\nD. This will help prevent dirt from getting on the soft, rubber surface of the shoes and causing blisters. [title] Run a hard-shoe brush over shoe soles twice a week for a few minutes to get rid of built-up dirt.","prompt":"[header] How to clean merrell shoes [title] Brush off small amounts of dirt regularly. [step] As merrell shoes are often used for outdoor activities, they're prone to building up dirt and other debris. Make a habit of cleaning merrell shoes after each use.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5683,"error":null,"output":"A"}
{"question_id":"hellaswag:00009795","item_index":93,"attempt":0,"prompt_hash":"6f3b8b58130ad5751d5653ac0e54c5134f62483be6b2c4d5e81ad0d86b03d5a5","question":"[header] How to tell if it's cubic zirconia and to see the best quality [title] Look at the color. [step] Loose cubic zirconia stones are generally colorless, but many shades of colors can be added using trace elements during the manufacturing process. [title] Check the clarity.\n\nA. [step] Clarity is considered lesser valuable than heavy metal, so it is very important to know how the stone is sold at consumers. [substeps] Never put gemstones or gemstones on well used stones.\nB. [step] These are usually colored by the highest brand of dye and with minimal intermediate spectrum radiation. The clarity is usually smooth and uneven throughout the dye.\nC. [step] In order to tell if the stones are blue, blue, or black, look at the lower tint of the pebbles. Some quartz stones have a greenish tint.\nD. [step] Clarity, or clearness, of loose cubic zirconia is a quality factor. The manufacturing process can sometimes cloud a stone, making visible imperfections.","prompt":"[header] How to tell if it's cubic zirconia and to see the best quality [title] Look at the color. [step] Loose cubic zirconia stones are generally colorless, but many shades of colors can be added using trace elements during the manufacturing process. [title] Check the clarity.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5684,"error":null,"output":"D"}
{"question_id":"hellaswag:00009258","item_index":94,"attempt":0,"prompt_hash":"ad96ef6a6794605e0e1618386065085b4e86656a1206779620b450b326a829b4","question":"[header] How to do undercut hair for men [title] Choose the right clipper for the person's hair. [step] Without a good clipper, doing undercut hair for men can be difficult. Ideally, your clipper will be a single-length clipper.\n\nA. While you may not necessarily be able to get a good cut on top or side of your head, you can train your mate to wear a clipper that matches his hair color. [substeps] Different clippers fit different bodies, but a clipper with a straight handle or one that has a square blower will give you more control.\nB. But if you want to do fading and tapering, you will need a clipper that allows length adjustments. [substeps] The ideal clipper to do undercut hair for men is one of high quality and high durability.\nC. Choose a clipper with a trimmer, or a medium length clipper. [substeps] If the person has long, thick hair, choose a clipper with a trimmer (or be gentle with the blade of the clipper, though) to minimize unnecessary damage.\nD. Start with a standard length, if possible, to ensure that you get the best results. [substeps] If you notice that there are chunks of hair sticking out between the length of the clipper and the direction your hair is heading, a clipper can help minimize the chunks.","prompt":"[header] How to do undercut hair for men [title] Choose the right clipper for the person's hair. [step] Without a good clipper, doing undercut hair for men can be difficult. Ideally, your clipper will be a single-length clipper.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5683,"error":null,"output":"B"}
{"question_id":"hellaswag:00006267","item_index":95,"attempt":0,"prompt_hash":"ba11e1f98b31c8575c30ef702bcaa72d917d3456fc40ba9c2ad010d0389dbb15","question":"[header] How to help a pregnant teen friend [title] Explore her choices. [step] Talk with her about the options she has and her feelings about each of them. Talk about each option without judgement or pressure as to what she should choose.\n\nA. [substeps] For example, if she wants to go to her doctor for an abortion and a friend wants to go, keep it brief. Reassure her that the decision is yours, not whether she should have to decide which one.\nB. This is her body, her pregnancy, and she is going to be the one who has to deal with the consequences, not you. [substeps] She could decide to have an abortion.\nC. This helps your friend learn if she would want to adopt a child, how to care for her baby, and just how the process works. [substeps] During this time, listen to your friend.\nD. If you believe there are going to be alternatives to her choice, take her in for your talk. [substeps] Give her some suggestions on which options you think she might be more successful in.","prompt":"[header] How to help a pregnant teen friend [title] Explore her choices. [step] Talk with her about the options she has and her feelings about each of them. Talk about each option without judgement or pressure as to what she should choose.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5684,"error":null,"output":"B"}
{"question_id":"hellaswag:00006178","item_index":96,"attempt":0,"prompt_hash":"c62efbcfccf4c1c493d90a4a4ae3dd49664c19f68e530ec063f97ed33a1c480a","question":"[header] How to plan a visit to the mission beach, san diego [title] Decide when to visit san diego. [step] September and october are considered peak season to enjoy at the mission beach. During this time, you can enjoy a pleasant weather with clear days.\n\nA. Early april through september is considered the peak season, so saturday is often the peak season. [substeps] If you're the type who believes in the views, seeing a part of san diego that's similar to the san diego theme park is definitely a would-be tourist experience.\nB. Of course, a month or two may not be enough time to enjoy the resort. [substeps] If you're going for a walk, consider visiting san diego during the week during the summer because the summer temperatures generally drop throughout the week.\nC. [title] Choose the medium of transportation and book your tickets in advance as sept-oct is the peak season and there is always a rush. [step] It might be difficult to arrange for tickets at short notice.\nD. [title] Decide how you are going to travel to the mission beach. [step] San diego is home to several different attractions including : [substeps] Whitewater rafting outings: these are short, scenic outings where you walk all the way to the beach.","prompt":"[header] How to plan a visit to the mission beach, san diego [title] Decide when to visit san diego. [step] September and october are considered peak season to enjoy at the mission beach. During this time, you can enjoy a pleasant weather with clear days.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5451,"error":null,"output":"C"}
{"question_id":"hellaswag:00006773","item_index":97,"attempt":0,"prompt_hash":"2bc654ace415a05070256ac9a359ee762636d25a709d4bcdbd70d014d6d184be","question":"[header] How to make black hair grow [title] Wash your hair every 4-7 days. [step] Washing regularly helps to remove any hair product build up you might have, allowing your scalp to breathe. When you wash, you also stimulate blood flow to the scalp, which encourages hair growth.\n\nA. You can soak a washcloth in shampoo and water, rub water in your hair, and rinse it. [substeps] Washing your hair every day also helps to keep your scalp healthy.\nB. Do not wash your hair less often or because you want to add volume and it can get dandruff. If you don't want to wash your hair every day, plan for your hair to get a little wet at the end of the day.\nC. [substeps] A lot of shampoos on the market contain sulfates, which can strip your hair of its natural oils. These natural oils help your hair to stay moisturized, preventing breakage.\nD. Use warm water at first, then let the water sit for 30 minutes before rinsing out the excess water. Once your hair is completely dry, apply conditioner to help it relax.","prompt":"[header] How to make black hair grow [title] Wash your hair every 4-7 days. [step] Washing regularly helps to remove any hair product build up you might have, allowing your scalp to breathe. When you wash, you also stimulate blood flow to the scalp, which encourages hair growth.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5451,"error":null,"output":"C"}
{"question_id":"hellaswag:00001325","item_index":98,"attempt":0,"prompt_hash":"15a034effe0a54ce51de24ff3f0194d1a8acdce91d202e80c0f53ce7440f53fa","question":"A blurry black stage is present and then all of a sudden a woman appears playing the piano. While the woman is playing, the blind artist stevie wonder appears playing the harmonica into his microphone. as he\n\nA. plays, the rest of his band joins in and the guitarist joins the woman on the piano.\nB. continues to play ,.\nC. plays, the woman continues to play the piano and then to himself in the gym, alone during the session, he hands her back to her then stops playing and begins putting his earphones down.\nD. plays, a beside the infographics shown in the screen and he fades away.","prompt":"A blurry black stage is present and then all of a sudden a woman appears playing the piano. While the woman is playing, the blind artist stevie wonder appears playing the harmonica into his microphone. as he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5457,"error":null,"output":"A"}
{"question_id":"hellaswag:00007607","item_index":99,"attempt":0,"prompt_hash":"fa0cab8ee537ac5366293b773f8e3213bd7292490d0936abc191f96f8a4c5c31","question":"[header] How to protect garden berries [title] Buy bird netting. [step] Bird netting has ¼ inch holes, making it near impossible for a bird or squirrel to get into the netting. Use commercial bird netting made of plastic or nylon.\n\nA. [substeps] Take it out of the bed whenever possible. You can use a wide spread (like your kitchen floor) or taller, more comfortable netting.\nB. This does not provide any protection against birds. Whenever possible, buy a bird net from a plant nursery or garden supply store.\nC. [substeps] You can use clear glass or plastic child-proof netting with thick sides for a very unique privacy. Sunlight will cause the netting to lose its leaves and will let in light, though it will not cover up the garden in direct sunlight.\nD. You can find bird netting online or at your local hardware store. [substeps] Buy enough bird netting to cover the entire berry plant, or row of berry plants, in your garden.","prompt":"[header] How to protect garden berries [title] Buy bird netting. [step] Bird netting has ¼ inch holes, making it near impossible for a bird or squirrel to get into the netting. Use commercial bird netting made of plastic or nylon.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5455,"error":null,"output":"D"}
{"question_id":"hellaswag:00007534","item_index":100,"attempt":0,"prompt_hash":"8ecd375706e9b4b036a4087e52f55594e8571a2d9c3008b327332e889fc7ede2","question":"[header] How to curl your hair with paper bags [title] Dampen your hair. [step] You can either create these curls after a shower, or just use a spray bottle to dampen your hair. It's important that you work with damp hair, rather than wet hair.\n\nA. The damp hair will mold to the hand of the bag. [title] Place the bag or sheet containing the hair and curls in a sock-style or small plastic container.\nB. If your hair is too wet when you begin, the paper will soak up the water and will not hold your hair properly. [substeps] For african american hair, start with dry hair.\nC. When the hair is dripping wet, apply a generous amount of spray to your hair. Then, let it air dry for several minutes before you proceed to curl.\nD. Simply wet your hair, then lightly comb your hair from roots to tips. [substeps] The goal of this method is to achieve roughly the same amount of curls as you would with dry hair.","prompt":"[header] How to curl your hair with paper bags [title] Dampen your hair. [step] You can either create these curls after a shower, or just use a spray bottle to dampen your hair. It's important that you work with damp hair, rather than wet hair.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5454,"error":null,"output":"B"}
{"question_id":"hellaswag:00007279","item_index":101,"attempt":0,"prompt_hash":"cc4b4d4decb8427e361ad7863f08a56da0338720a9f739c3ac9b868e36c08149","question":"[header] How to give yourself a french manicure using tape [title] Rub a cotton swab with nail polish remover all of your nails. [step] Not only will this remove any chipped or leftover polish, but it will also remove any moisturizer from your nail. If you have a moisturizing agent, such as lotion or oil, on your nail, the polish will not stick properly.\n\nA. [title] Put the polish remover in a small bowl. [step] You will need it in a few minutes to get this solution on your toes.\nB. [title] Spread a moisturizing powder all over all of your nails, using circular motions until you come into contact with your skin. [step] Always buff your nails before you start.\nC. [title] Lather the cotton in the polish remover. [step] Use a soft washcloth to pick the polish up.\nD. Make sure that you've created a perfect canvas for your french manicure. [title] Apply your base color to the entire nail.","prompt":"[header] How to give yourself a french manicure using tape [title] Rub a cotton swab with nail polish remover all of your nails. [step] Not only will this remove any chipped or leftover polish, but it will also remove any moisturizer from your nail. If you have a moisturizing agent, such as lotion or oil, on your nail, the polish will not stick properly.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5455,"error":null,"output":"D"}
{"question_id":"hellaswag:00000220","item_index":102,"attempt":0,"prompt_hash":"f8df21ac2ea533a54bd82b4fa42b604b026a59f927f3e93caacee714ab32d02f","question":"She forgets the scissors and walks out to grab them and comes back to apply more paper on the wall. a break\n\nA. is needed and the woman get a drink from the kitchen.\nB. artist rolls up her paper so it's more in line with the wall paper cover back to cover the table.\nC. artist is using a tool over and over to shape the board that was sitting in the corner before again taking the leaflets and creating a new sheet of paper, this time incorporating another white sheet.\nD. room party is being held and she is shown sitting and talking and playing with a bobble doll.","prompt":"She forgets the scissors and walks out to grab them and comes back to apply more paper on the wall. a break","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5454,"error":null,"output":"A"}
{"question_id":"hellaswag:00004573","item_index":103,"attempt":0,"prompt_hash":"560d4fe8dd0394d395e993f199eab50bc03873a92a9ccd16eaa8bdc45caf9d6e","question":"[header] How to buy a flat iron [title] Determine your hair type. [substeps] Consider whether your hair is fine or thick. Fine hair requires less heat to straighten.\n\nA. Curly hair needs less curl in order to increase the volume, and heavy hair needs less curl in order to achieve a sleek, beach-like look. Generally speaking, medium, or thick hair should be naturally smooth and soft.\nB. Thick hair is much harder to straighten because the individual hairs are coarser, causing them to have less moisture. Those with curly or wavy hair should purchase an iron with a higher heat setting.\nC. Thick hair needs more dry heat so soft (but not very dry) hair will not straighten as quickly. Fine hair of all hair sizes, shapes, and hair types generally require a lot of straightening.\nD. Thin people take less heat to get what they want because the hair fibers are the stronger. Thin people usually avoid blow drying because their hair comes much more naturally when it is dry and soft.","prompt":"[header] How to buy a flat iron [title] Determine your hair type. [substeps] Consider whether your hair is fine or thick. Fine hair requires less heat to straighten.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5456,"error":null,"output":"B"}
{"question_id":"hellaswag:00006580","item_index":104,"attempt":0,"prompt_hash":"48da2cae803f07de2eb3c6f8b133dbf44ecf29e699568773d6f4fc1e3fb4ccf0","question":"[header] How to get your baby or child back [title] Find an attorney. [step] If your child has been taken out of your custody, you should seriously consider hiring a reputable family law attorney. Family law attorneys can specialize in many different disciplines (e.g., divorce, adoption, custody) so be sure you find one that specializes in juvenile court processes and custody issues.\n\nA. [substeps] For example, if your child is being adopted by another parent, the court might not consider an attorney who lives in your county. Accordingly, try to find an attorney with experience dealing with only juvenile court cases.\nB. To find a reputable attorney, call your local bar association or visit their website. State and town bar associations often have attorney referral services for your convenience.\nC. [substeps] If the opposing party is impartial and has a strong argument about custody, you should hire a civil rights attorney. He or she will be better equipped to support your case than a novice in custody or parenting.\nD. [substeps] Family law attorneys often provide limited legal services based on circumstance. For example, your child might not have a competent legal guardian if she were abandoned by her grandparents.","prompt":"[header] How to get your baby or child back [title] Find an attorney. [step] If your child has been taken out of your custody, you should seriously consider hiring a reputable family law attorney. Family law attorneys can specialize in many different disciplines (e.g., divorce, adoption, custody) so be sure you find one that specializes in juvenile court processes and custody issues.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5451,"error":null,"output":"B"}
{"question_id":"hellaswag:00008568","item_index":105,"attempt":0,"prompt_hash":"62f9f096714b7724d6db3f4d1ff3d1f6d71cd16d25c6b43d0cc7c32869866712","question":"[header] How to hump ponytail [title] Start with your hair down and part a section out about 1.5-2 inches (3.8-5.1 cm) wide by 1 inch (2.5 cm) in the front center of your head. [step] This size depends on your head and the results you want. [title] Put the parted hair in a rubber band or bobby pin.\n\nA. [step] If you have a ponytail, wrap the hair around the rubber band until it almost touches your scalp. Double down to make sure it is secure.\nB. [title] Put the rest of your hair in a pony as you would normally. [step] If you have shorter hair or flyaways you may want to use hair spray.\nC. [step] Pull your hair in to secure it and avoid letting it fall down into your face. [title] Repeat the process as much as needed to secure the hair in place.\nD. [step] Or you can use ponytail spray, any hair spray will do. [title] Move the plastic hair roll into the ponytail.","prompt":"[header] How to hump ponytail [title] Start with your hair down and part a section out about 1.5-2 inches (3.8-5.1 cm) wide by 1 inch (2.5 cm) in the front center of your head. [step] This size depends on your head and the results you want. [title] Put the parted hair in a rubber band or bobby pin.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5453,"error":null,"output":"C"}
{"question_id":"hellaswag:00006665","item_index":106,"attempt":0,"prompt_hash":"f04212d3483ee36a78493f35f6f1d1518cef2d84dffc2cc04bf13fd31a4812a3","question":"[header] How to have a good soccer practice [title] Focus on 1-2 skills or ideas each practice. [step] This is especially important with younger kids, but even older players benefit from focused practices. Don't try and teach individual defending, corner kick plays, shooting, and two new drills on the same day.\n\nA. However, if you find yourself thinking of them, ask yourself why. [title] Engage your core and your legs at low pressure regardless of whether they're based off of stronger, defensive moves.\nB. [substeps] When you're in the right state, you'll know you're improving fast because there isn't a way to overshoot and other backpull plays as you practice. [title] Understand how to play each soccer game.\nC. The more strategies you have in place to reinforce yourself, the more you'll begin to visualize your actions as well as countering them with each event. [substeps] Some common techniques used improve your technique no matter what sports you are, but practicing them regularly will make it easier for you to remember them.\nD. Instead, plan ahead of time to focus on one or two specific areas, allowing players to really concentrate on improving these areas. Some areas of practice naturally work well together, as well : [substeps] Crossing and aerial work (headers, clearances, marking in the box, etc.","prompt":"[header] How to have a good soccer practice [title] Focus on 1-2 skills or ideas each practice. [step] This is especially important with younger kids, but even older players benefit from focused practices. Don't try and teach individual defending, corner kick plays, shooting, and two new drills on the same day.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5449,"error":null,"output":"D"}
{"question_id":"hellaswag:00007018","item_index":107,"attempt":0,"prompt_hash":"e54f479b3203934a4b4d0a23739223c002c8b93b96878728b85c93f521c28820","question":"[header] How to determine your body shape [title] Pay attention to where you gain weight. [step] Each body type is prone to weight gain in a specific area-for example, the thighs or stomach. Know where you gain weight most often to clue yourself in about your shape.\n\nA. What parts of your body determine your body shape? [substeps] Body mass index (bmi). Bmi is your height and you weigh between 135 and 140 kilograms (35 and 200 kg).\nB. How does it affect your figure? [substeps] Be aware that your body size can also affect your weight. People with denser or heavier bodies are more likely to gain weight than people with a smaller or heavier body.\nC. [title] Stand in front of a mirror while wearing as little as possible. [step] Clothing can mask your shape, making it harder for you to get an accurate picture of your curves.\nD. [substeps] Young adults generally weigh about 170 pounds when they're in their mid-teens , 55 pounds when they're in their twenties, and up to 65 pounds at their maximum before adulthood. Men typically weigh 150 pounds when they start puberty, and at 40, the average for children aged 45 to 55 is 18.","prompt":"[header] How to determine your body shape [title] Pay attention to where you gain weight. [step] Each body type is prone to weight gain in a specific area-for example, the thighs or stomach. Know where you gain weight most often to clue yourself in about your shape.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5449,"error":null,"output":"C"}
{"question_id":"hellaswag:00010027","item_index":108,"attempt":0,"prompt_hash":"e954ba7b67354289f311b4ade1ed6a43ba9fee6baa6a7fe6a30bb85959eb8f53","question":"[header] How to keep your toddler from biting [title] Know that biting is normal. [step] Your toddler is biting to communicate with you. You must learn what your toddler is trying to say and then make adjustments.\n\nA. Wait patiently to look at the bite and respond to what your toddler is saying. Keep quiet and look at your toddler as he is talking to you.\nB. [substeps] Use positive reinforcement to encourage your toddler to bite. Do not hit your toddler, hit him, pull on his tail, or tease or ridicule your toddler's behavior.\nC. Although biting is normal, it is still not acceptable. Typical reasons for biting include : [substeps] Expressing a need or feeling such as anger, frustration, joy, or excitement being overwhelmed by their surroundings tired teething experimenting copying other children seeking attention [title] Determine why your child is biting.\nD. [substeps] Instruct your toddler to be quiet if she is biting. This will discourage your toddler from biting.","prompt":"[header] How to keep your toddler from biting [title] Know that biting is normal. [step] Your toddler is biting to communicate with you. You must learn what your toddler is trying to say and then make adjustments.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5448,"error":null,"output":"C"}
{"question_id":"hellaswag:00006469","item_index":109,"attempt":0,"prompt_hash":"99a10b7844605632eaf4c729e57ecfc9e35b433b59b624cbda5c87982556c4f3","question":"[header] How to wear blue lipstick [title] Opt for navy. [step] Navy blue lipstick is flattering on almost everyone. It is also the most popular and easy to find color.\n\nA. It is best picked for a feminine look and is fresh and bright with aqua accents. [substeps] Navy lipstick is also the most popular and easy to find color.\nB. If you don't want to go all out, then go for a brighter shade. You can even always choose lipstick with a hint of bright red.\nC. [substeps] Choose a soft, neutral color, like light blue or taupe. Avoid bright colors like red, orange, and green as these go against the creamy body and choices.\nD. If you are new to blue lipstick, then a navy shade might be the best place to start. [substeps] Navy lipsticks are usually matte, but you can apply a clear gloss over the lipstick if you prefer a shiny lip look.","prompt":"[header] How to wear blue lipstick [title] Opt for navy. [step] Navy blue lipstick is flattering on almost everyone. It is also the most popular and easy to find color.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5454,"error":null,"output":"D"}
{"question_id":"hellaswag:00003236","item_index":110,"attempt":0,"prompt_hash":"b6ebcac044236e3ae29eb9c1c2d0dd1fc0e8426b277ca9961bcbcb901981e7b3","question":"We see more asian art and a man with a sword followed by more art and landscapes. A statue sits outside surrounded by trees. we\n\nA. see the man with a cello.\nB. then see further images of dirt and castles.\nC. see more landscapes and title screens.\nD. see a letter on the screen and see people practicing martial arts and more images of paintings.","prompt":"We see more asian art and a man with a sword followed by more art and landscapes. A statue sits outside surrounded by trees. we","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5454,"error":null,"output":"C"}
{"question_id":"hellaswag:00007964","item_index":111,"attempt":0,"prompt_hash":"f4bc3b7ebb7df7135b6a8e5727d08991a5289c8c83a0dff683edeceb4856f0b2","question":"[header] How to get a free room upgrade in las vegas [title] Visit vegas at the right time. [step] The dates of your visit to vegas may increase your chances of receiving a free room upgrade. Travel experts, however, have conflicting opinions on the \" right time \" to visit vegas.\n\nA. [substeps] It's highly likely that your hotel will provide a free room upgrade for several days prior to the date of your visit. However, las vegas law states that about 30 days before you visit, when a party or other event places a hotel upgrade fee as a matter of course, at 100.00.\nB. [substeps] Visit during a period of low occupancy. According to some travel experts, if you want a free upgrade, reserve a room during a period of low-occupancy.\nC. The only person who can confirm a free room upgrade is you-and only you. Keep in mind that visitors to vegas may not agree to a free room upgrade just yet, though, so it may be a good idea to bring along helpful travel proof documents to prove your case and correct any misconceptions.\nD. To prevent this from happening, get your bearings on the weekend before your visit so you don't miss any opportunities to adventure. [title] Search online to find the free ballroom's features and typical number of rooms remaining on the school spectrum.","prompt":"[header] How to get a free room upgrade in las vegas [title] Visit vegas at the right time. [step] The dates of your visit to vegas may increase your chances of receiving a free room upgrade. Travel experts, however, have conflicting opinions on the \" right time \" to visit vegas.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5452,"error":null,"output":"B"}
{"question_id":"hellaswag:00001869","item_index":112,"attempt":0,"prompt_hash":"30c83d2855624e0484cc7616a31aff30a6ec2655ca5e8f5f57e4f0615fe4986c","question":"A man is standing in front of a window playing a violin. medications\n\nA. are shown on a table.\nB. are being given to his stomach.\nC. are dispensed to the boy while he is playing the acoustic instrument.\nD. are being given to the man on the violin.","prompt":"A man is standing in front of a window playing a violin. medications","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5453,"error":null,"output":"A"}
{"question_id":"hellaswag:00001757","item_index":113,"attempt":0,"prompt_hash":"59e8ceb82bb758f35862e0cc78c7bce1b0bc5c9364a956f5b460331cb02d039a","question":"Next, an actual cheerleaders begins talking and you can see her getting dressed and someone is doing her hair. more teams and members\n\nA. are shown playing games of tug of war with other teams.\nB. appear and they all are practicing jumps, steps, and going over techniques.\nC. of the team wearing the bonengyr are shown and the d cheerleader very excited about it.\nD. are seen interacting with one female and go walkway via another.","prompt":"Next, an actual cheerleaders begins talking and you can see her getting dressed and someone is doing her hair. more teams and members","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5455,"error":null,"output":"A"}
{"question_id":"hellaswag:00003882","item_index":114,"attempt":0,"prompt_hash":"1de01d933e869e381030cc91009e585d11e2dca1b7339411be63e9a88321d646","question":"[header] How to use hypnosis to overcome challenges in your life [title] Reflect on the challenges in your life. [step] Before you can really address any problems or challenges, you need to be able to address that they are real. Think about the various hurdles in your life and how they impact you and your family.\n\nA. In case you or your loved ones are struggling with or overlooking something, any source of solace you have is good. [substeps] You or your loved ones are bound to have challenges.\nB. Are you facing challenges when you don't use hypnosis, or do you deal with them in an ideal situation? [substeps] Consciously using hypnosis may help you identify where in your life those challenges are. [title] Assess how much you love the results of hypnosis.\nC. Do you suffer from a difficult psychological issue? Do you battle with addiction or impulse control? Once you have identified your problems, you can begin working towards managing them. [substeps] Talk with your friends and family about your challenges and how best to deal with them.\nD. [substeps] Write down all the obstacles in your life, how they affect your relationships with family and friends, and the ways that you deal with them. You might record family problems, list food allergies, or other specific challenges that you are facing as problems.","prompt":"[header] How to use hypnosis to overcome challenges in your life [title] Reflect on the challenges in your life. [step] Before you can really address any problems or challenges, you need to be able to address that they are real. Think about the various hurdles in your life and how they impact you and your family.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5452,"error":null,"output":"C"}
{"question_id":"hellaswag:00007513","item_index":115,"attempt":0,"prompt_hash":"1409ac3d09c3e4098dbb4878d4e69a7e4f68447110609511d110bda2b29f4fb8","question":"[header] How to baker act someone [title] Educate yourself about the baker act. [step] The baker act, also known as the florida mental health act (florida statute chapter 394, part i) is a law that was passed to ensure and regulate emergency mental health treatment services for those suffering from a mental illness. This law is meant to protect all people with serious mental illnesses in the state of florida.\n\nA. [substeps] To become a baker, you must have completed a federal or equivalent training program. There are a number of courses available for the united states that will help you get into baker school.\nB. [substeps] The baker act provides procedures to guide both voluntary admissions and involuntary examinations of people suffering from a mental illness. The act also provides procedures for the involuntary inpatient placement (civil commitment) for someone suffering a mental health crisis.\nC. One of the biggest reasons you would be in danger is if your family has suffered from mental illness. Having health care providers from the civil rights commission will help make your situation better.\nD. It can also help if you are suffering from a negative mental illness in order to manage mental health. [substeps] To read about this writer, see take examiners under the new york real world, which rules provide the baker act.","prompt":"[header] How to baker act someone [title] Educate yourself about the baker act. [step] The baker act, also known as the florida mental health act (florida statute chapter 394, part i) is a law that was passed to ensure and regulate emergency mental health treatment services for those suffering from a mental illness. This law is meant to protect all people with serious mental illnesses in the state of florida.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5454,"error":null,"output":"B"}
{"question_id":"hellaswag:00004406","item_index":116,"attempt":0,"prompt_hash":"91dc6e707e5bdd53e5d00c2a8ffb6593675210593e3f1899e338f686f566e1bc","question":"[header] How to wear leather leggings [title] Choose between real and faux leather. [step] Most leather leggings are actually made with faux leather instead of the real deal, but you can go with either option. [substeps] Most faux leather is made from stretchy fabric, like spandex, coated with some form of black polyurethane.\n\nA. The black material forms a good barrier against excess heat and moisture, making it easy to wear. Add leather leggings to your wardrobe in order to keep them in style without leaving all or most of the ankle or thigh holes exposed.\nB. This creates a cushioning effect, allowing you to get a good, durable fit. As such, faux leather leggings are very flexible and comfortable.\nC. Faux leather is generally cheaper and easier to care for. You can usually wash faux leather in your washing machine without damaging the material, but real leather tends to be more temperamental.\nD. This covering means that the leggings have a highly visible sheen and an ultra-reflective layer on them. This makes them a classic choice for casual situations.","prompt":"[header] How to wear leather leggings [title] Choose between real and faux leather. [step] Most leather leggings are actually made with faux leather instead of the real deal, but you can go with either option. [substeps] Most faux leather is made from stretchy fabric, like spandex, coated with some form of black polyurethane.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5453,"error":null,"output":"C"}
{"question_id":"hellaswag:00003738","item_index":117,"attempt":0,"prompt_hash":"011364c7e8659d158b6f869bb6013ed6cd9c3dee8b963a430bcac857a829dd2a","question":"[header] How to remove iron on patches [title] Check to see if your item can take the heat. [step] Unless you added the patch yourself, you will first need to make sure your item won't be damaged by ironing it. Not all \" iron on \" patches are actually applied with heat.\n\nA. Determine if your item can take the heat without removing the iron. [title] Remove any adhesive (if you have any).\nB. [substeps] The simplest way to tell if your item can take the heat is to check with a professional iron-on patch test kit. Over the counter formulas are known to require precision.\nC. Patches available from home, the factory channel, and websites are the ones most commonly need to be ironed. Check your item's version of the iron, especially if you must use regular iron sheets.\nD. [substeps] Pick a small, inconspicuous part of the item not easily visible during normal use. Place wax paper or a thin dish towel on top of the area.","prompt":"[header] How to remove iron on patches [title] Check to see if your item can take the heat. [step] Unless you added the patch yourself, you will first need to make sure your item won't be damaged by ironing it. Not all \" iron on \" patches are actually applied with heat.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5453,"error":null,"output":"D"}
{"question_id":"hellaswag:00006615","item_index":118,"attempt":0,"prompt_hash":"5c8c0dbab8756881eb0d5c30916c2c838c537b8297d59d0df8745e64f9b83639","question":"[header] How to make a sterile bacterial smear [title] Remove all unneeded equipment and objects from the lab bench. [substeps] Leaving objects on the lab bench can get in the way or be a safety hazard. Paper left on the bench can catch fire if a bunsen burner is knocked over.\n\nA. [title] Squirt liquid disinfectant onto the lab bench. [substeps] Use a liquid disinfectant meant for hard surfaces, containing bleach or ethanol.\nB. Leaving objects off your bench is the key to your success. Removing one item from the lab bench is not recommended unless you want to collect germs.\nC. Put any food or also any cutlery or other items that may have been sitting on the bench away from where the patient is working. Sanitize hands, feet, clothing, and anything else in the area that would be touched by the patient.\nD. Certain equipment, like gloves, will drop out of the solution and become a fire hazard and potentially lead to serious illness and death. Important objects to remove include any infected equipment you may have, poultry or other food items such as fruit or vegetables.","prompt":"[header] How to make a sterile bacterial smear [title] Remove all unneeded equipment and objects from the lab bench. [substeps] Leaving objects on the lab bench can get in the way or be a safety hazard. Paper left on the bench can catch fire if a bunsen burner is knocked over.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5257,"error":null,"output":"A"}
{"question_id":"hellaswag:00001350","item_index":119,"attempt":0,"prompt_hash":"c3543478227425dba91925a468e0a2f0d48a955f9c5dd270dec16092677ff247","question":"People are standing on a basketball court. A man in an orange shirt throws a ball. they\n\nA. stand around and talk.\nB. start playing a game of basketball.\nC. take a break and the game resumes.\nD. run around doing high kicks.","prompt":"People are standing on a basketball court. A man in an orange shirt throws a ball. they","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5256,"error":null,"output":"B"}
{"question_id":"hellaswag:00009962","item_index":120,"attempt":0,"prompt_hash":"84100849c7ccc63893b2329f13708055ead9cd99bae269cfaab4701131a48967","question":"[header] How to check out a gym daycare [title] Ask your local gym if they offer daycare services. [step] One way you can locate a gym daycare is to inquire if your regular gym has daycare services. Perhaps they have a daycare area in the back that you never noticed before.\n\nA. [title] Inquire about annual discount on a regular gym visit. [step] While most gyms provide a discount, that doesn't mean the payout won't be good for the gym.\nB. If they do, ask them if you can come and help. Not everyone has this option, and many gyms only offer this option in certain circumstances.\nC. Or maybe they recently expanded to include a daycare for their clients. Speak to a representative at your local gym to see if you can place your child in a daycare there.\nD. Call their toll free number at 1-800-272-1211 and inquire whether they offer daycare services. If they don't, look them up on their website to find out.","prompt":"[header] How to check out a gym daycare [title] Ask your local gym if they offer daycare services. [step] One way you can locate a gym daycare is to inquire if your regular gym has daycare services. Perhaps they have a daycare area in the back that you never noticed before.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5257,"error":null,"output":"C"}
{"question_id":"hellaswag:00009019","item_index":121,"attempt":0,"prompt_hash":"9b65ad68a0672485ca7c2d8cc4202cd368a686e44dd0aa89d702faa64a61741a","question":"[header] How to get workers compensation insurance [title] Determine whether a business must purchase workers' compensation insurance. [step] Each state administers its own workers' compensation program. The purpose behind workers' compensation insurance is to give injured or ill workers immediate financial relief for at least a portion of their wages while they are unable to work.\n\nA. As a result, employees may have to invest a significant portion of their total expected wage in the work they do. In some areas, the amount of time that employee must have workers' compensation insurance is less than the amount of wages they generate on a daily basis in a single year.\nB. [substeps] In order to determine whether an employer must buy employees' compensation insurance, it must first determine whether a business must purchase workers' compensation insurance. When another business brings workers' compensation insurance to him, the employer may wonder what sort of policy will help him out.\nC. As a general rule, any business that hires employees who are not also owners of the business must obtain workers' compensation insurance. [substeps] There may be severe penalties for businesses or employers who fail to maintain workers' compensation insurance for their employees as required.\nD. States may require workers' compensation insurance based on the business' economic needs. [substeps] States such as maryland provide the premium on worker's compensation insurance for businesses over 45 years of age and older.","prompt":"[header] How to get workers compensation insurance [title] Determine whether a business must purchase workers' compensation insurance. [step] Each state administers its own workers' compensation program. The purpose behind workers' compensation insurance is to give injured or ill workers immediate financial relief for at least a portion of their wages while they are unable to work.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5257,"error":null,"output":"C"}
{"question_id":"hellaswag:00003224","item_index":122,"attempt":0,"prompt_hash":"84b31f596e55b21c9c1996b228dab5446c7bf99cb2292058210f0046398cf0c4","question":"Two women are seen looking at the camera with one speaking and holding a hair dryer. the woman\n\nA. shows off the hair dryer while speaking and pointing to the woman in the chair.\nB. turn the dryer on and begins pressing all the hair down into the dryer.\nC. long the hair and then begin blow drying the hair with her hands.\nD. then begins curling one part of the hair while the other shows her how to use the dryer.","prompt":"Two women are seen looking at the camera with one speaking and holding a hair dryer. the woman","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5256,"error":null,"output":"A"}
{"question_id":"hellaswag:00001129","item_index":123,"attempt":0,"prompt_hash":"d7acf8a87def9dc1d94eff9ac20e95a4f1297b078ac065103a68e008878b2eee","question":"An introduction comes onto the screen for a video about fishing lures. Several men show off the different lures they are using for ice fishing. the action of the lures\n\nA. is shown with people enjoying fish in the area.\nB. successfully catches several fish before submerging into the warm waters.\nC. providing food and protection kicks in and then an introduction comes onto the screen for a news segment.\nD. is shown underwater as several different fish go after the lures.","prompt":"An introduction comes onto the screen for a video about fishing lures. Several men show off the different lures they are using for ice fishing. the action of the lures","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5267,"error":null,"output":"D"}
{"question_id":"hellaswag:00006088","item_index":124,"attempt":0,"prompt_hash":"79dee39e00b1476240f4bbe6a003e1553ae47a1c5e564e89da8706a7cf0fb480","question":"[header] How to be a good guest (for mothers in law ) [title] Rent a car, or have an alternate way to get around. [step] Don't expect your hosts to cart you around to wherever you want to go. Additionally, your hosts may be not having their vacations as you are and working throughout the day-the last thing they want to do when they get home is drive you places.\n\nA. [title] Plan other things during your visit, and let them know in advance. [step] If they are working during your stay, they may need some time to themselves away from' entertaining'.\nB. Likewise, don't expect to be away much. Leaving several random nights of your period in your room is a waste of time, the worst thing ever.\nC. It's best to try any alternate route so as to avoid a poor rental. [title] Look through a phone book, internet guide, or library.\nD. [substeps] Be sure you bring an alternate route. [title] Buy a gym membership (for women) and workout gear.","prompt":"[header] How to be a good guest (for mothers in law ) [title] Rent a car, or have an alternate way to get around. [step] Don't expect your hosts to cart you around to wherever you want to go. Additionally, your hosts may be not having their vacations as you are and working throughout the day-the last thing they want to do when they get home is drive you places.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5267,"error":null,"output":"A"}
{"question_id":"hellaswag:00001660","item_index":125,"attempt":0,"prompt_hash":"a9d4c5d9e15458361d2d5316ffca07a042bbe38d81ef5c52a0549738db76f03f","question":"A man plays the tam-tams in a room. The man plays only in one tam-tam. then\n\nA. , the man plays only in the third tam-tam.\nB. , the man plays only one tam-tam in a room.\nC. , the man play only in three tam-tam.\nD. , the man plays the two tam-tams.","prompt":"A man plays the tam-tams in a room. The man plays only in one tam-tam. then","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5265,"error":null,"output":"D"}
{"question_id":"hellaswag:00006341","item_index":126,"attempt":0,"prompt_hash":"6980be1474583497889ec29ebc4fc04da1e321df9cf98b6b75aa1fc8bc1f224c","question":"[header] How to get rid of dark lips [title] Use make-up to lighten your lips for the night. [step] Whether for a party, costume, or new \" look, \" you can make-up to brighten your lips without causing long term changes or damage. [substeps] Try out lighter lip sticks colors, like peach or coral.\n\nA. Talk to your lip-care professional about what kinds of lipstick work best and what you might like your lips to have before taking a night off. Dark lipstick will dry out your lips and keep them from being used for daytime interactions, including a costume party.\nB. Use lip gels and clear lip gloss to add moisture, stay in place throughout the night, and reassure your skin that the brightening effect is beginning. You'll also want to consider applying concealer under your lips, where breakage is less, and concealer beneath your eyelids.\nC. Other colors color your lips for the night out, rather than just one. No makeup works as well, but a warm lipstick, eyeshadow color or lip glomp might or may lighten your lips for the night out.\nD. Use powder or concealer to lighten lips a little bit at a time. [title] Mix and match powder and concealer with your lip gloss to a more specific tint.","prompt":"[header] How to get rid of dark lips [title] Use make-up to lighten your lips for the night. [step] Whether for a party, costume, or new \" look, \" you can make-up to brighten your lips without causing long term changes or damage. [substeps] Try out lighter lip sticks colors, like peach or coral.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5267,"error":null,"output":"D"}
{"question_id":"hellaswag:00002511","item_index":127,"attempt":0,"prompt_hash":"37fc8988e560a7d1d0f4f73f4fe02776b76a41abfb67a45f0f3acd81b61ad7ab","question":"A group of people are sitting in their donut tubes up on a snowy mountain. they\n\nA. snow tube down the mountain together.\nB. get into them and ride down the slopes.\nC. are going down the slope very quickly.\nD. are climbing, riding in the tubes, passing several waterfalls.","prompt":"A group of people are sitting in their donut tubes up on a snowy mountain. they","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5264,"error":null,"output":"A"}
{"question_id":"hellaswag:00007324","item_index":128,"attempt":0,"prompt_hash":"d94ecffbf449517b6067da4be9445bf2cffa288fc5424b445be0e0fecd770c58","question":"[header] How to do ombre eye makeup [title] Prime your eyes. [step] It is important to prime your eyes to prevent creasing. You can use the makeup primer of your choice, or if you don't have primer, you can cover your eyelids with concealer and powder.\n\nA. While you will have to use the primer to soften your eyes, it will provide you with heavier coverage. [substeps] Ask a friend or family member to prime you if you are not sure how to do it.\nB. Use a small brush to prime your eyes. Do this for one hour, according to the directions on the packaging.\nC. Use a small amount of concealer to cover your eyelid and then cover it in a nude powder. Or, you can use a nude cream eye shadow.\nD. [substeps] Then, in an empty makeup-jar, two 1 oz. ( 237 ml) makeup or powder primer (without the one you used to prime your eyelids), two 1 oz.","prompt":"[header] How to do ombre eye makeup [title] Prime your eyes. [step] It is important to prime your eyes to prevent creasing. You can use the makeup primer of your choice, or if you don't have primer, you can cover your eyelids with concealer and powder.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6048,"error":null,"output":"C"}
{"question_id":"hellaswag:00009755","item_index":129,"attempt":0,"prompt_hash":"79cb51a76507f743b80f5f657ccdebfbdf4dd9f5d4d94e61673bbbc60bf3ad3e","question":"[header] How to accentuate your tan [title] Wear white. [step] White is the most common color worn to accentuate a tan. It is the best color to make your skin appear darker than it really is.\n\nA. Bright white is best for cool skin tones. Off-white is best for naturally golden skin tones.\nB. Pinks, yellows, and violets, on the other hand, seem paler and more vibrant. If you're not a fan of pink or golds, then a matching white spot is probably out of place.\nC. It adds a bit of refinement to your complexion and wrinkles up your body. With white, your tan doesn't have to be flawless.\nD. [substeps] If you want to draw attention to your skin you might opt for an updos or edgy white. A white is also the best shade of skin to highlight a tan as it hides dark circles and adds intensity.","prompt":"[header] How to accentuate your tan [title] Wear white. [step] White is the most common color worn to accentuate a tan. It is the best color to make your skin appear darker than it really is.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6054,"error":null,"output":"A"}
{"question_id":"hellaswag:00002334","item_index":130,"attempt":0,"prompt_hash":"00c1b816c7b9c837882b633b463a2ccd584552da577276867e7be36447074d1b","question":"A still shot is shown as the word honing is on the screen and music plays in the background. he\n\nA. is now entering a building and is talking about vacuuming the floor.\nB. has equipment and tools for shoots, some above ground, others smoke and still shots of himself taking off with his shot put.\nC. finally uses a hammer to hammer the window and it sticks in place with ease.\nD. explains how to hold a honing steel.","prompt":"A still shot is shown as the word honing is on the screen and music plays in the background. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6052,"error":null,"output":"D"}
{"question_id":"hellaswag:00002084","item_index":131,"attempt":0,"prompt_hash":"cea50f3cf5d3baa29b5034c2ab7b7181804e5a87049745adcbfb7ccbb02c050c","question":". a montage of athletes completing incredible stunts and races\n\nA. are shown as well as hundreds of people watching in the audience.\nB. around a track.\nC. is shown with the screen off for a while.\nD. is show before ending with images of others.","prompt":". a montage of athletes completing incredible stunts and races","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6051,"error":null,"output":"A"}
{"question_id":"hellaswag:00004972","item_index":132,"attempt":0,"prompt_hash":"771c43e251e3054f6fe97bb45f3f5ad43b488000a8bb5ecc4faecf9fd038e655","question":"[header] How to use the inversion method to grow hair [title] Choose your oil. [step] There is not one agreed upon oil for the inversion method. Try olive oil, castor oil, coconut oil, grape seed oil, or moroccan argon oil, if you want to use natural ingredients.\n\nA. In most countries, commercial oil will not be included in the treatments. [substeps] Choose an oil based on an herb called mint, chamomile oil, or chickweed oil.\nB. [substeps] Choose an oil with an aroma that you find soothing. Since the inversion method incorporates massage, pick a scented oil that you find relaxing.\nC. [substeps] Adults need one/8 cup (55 grams) of oil for every ten pounds you're losing. If you're starting breastfeeding or are doing surgical work, make sure to think about oil preferences.\nD. However, you should definitely make sure to use other oils that are suitable for this experiment. [substeps] Avocado oil has been shown to thicken hair faster than olive oil because of the more concentrated oil used.","prompt":"[header] How to use the inversion method to grow hair [title] Choose your oil. [step] There is not one agreed upon oil for the inversion method. Try olive oil, castor oil, coconut oil, grape seed oil, or moroccan argon oil, if you want to use natural ingredients.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6047,"error":null,"output":"B"}
{"question_id":"hellaswag:00009700","item_index":133,"attempt":0,"prompt_hash":"f389d4539380ec235cbc79970aa9d1d210f764403853e646dcc97dc6b5f5f872","question":"[header] How to get an itemized list of costs from your attorney [title] Ask about itemized bills during your consultation. [step] Before hiring an attorney, you should schedule a consultation. At the consultation, you can ask a variety of questions, including about fees.\n\nA. Also discuss the attorney's experience handling cases such as yours. [substeps] The attorney should be honest with you.\nB. For example, try to be as specific as possible when talking about premiums. [substeps] Before booking the consultation, make sure the attorney can accommodate your needs.\nC. You should ask whether the lawyer will provide you with an itemized bill and whether it will increase your costs. [title] Make sure the lawyer agrees to provide itemized bills.\nD. You should also ask about what services the attorney will charge. Some fees might include : [substeps] Client alimony other phone or customer service fees for hr a child custody and visitation schedule [title] Talk to the attorney about how to reach a settlement.","prompt":"[header] How to get an itemized list of costs from your attorney [title] Ask about itemized bills during your consultation. [step] Before hiring an attorney, you should schedule a consultation. At the consultation, you can ask a variety of questions, including about fees.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6044,"error":null,"output":"C"}
{"question_id":"hellaswag:00007801","item_index":134,"attempt":0,"prompt_hash":"f8d7c908a876369e7a7bd1b13afde32db48e684995dbbfccb13bddf6ccef6fa6","question":"[header] How to get constellation piercings [title] Ask friends or family who have piercing. [step] To find a good piercing artist, you can start by asking others about their piercings and who did them. Perhaps you have a friend who has a beautifully done piercing and you ask them for a referral to their artist.\n\nA. If you make this list, you'll be able to find a talented artist. [substeps] Ask for a recommendation from other friends or family members that have pierced their own piercings.\nB. Or maybe you have a family member who had a positive experience and is willing to refer you to the artist for your piercing. [substeps] You may focus on asking friends or family who have constellation piercings already that you like or find appealing.\nC. Or you may reach out to someone who has pierced their own basic piercing. Make sure that you get the information that you need about piercing artists in order to get an accurate picture of how well they can care for their piercings.\nD. Or perhaps you have a friend who just pierced herself? All of these may help you improve your technique. [substeps] You may ask some of your friends for a referral to your tattoo artist.","prompt":"[header] How to get constellation piercings [title] Ask friends or family who have piercing. [step] To find a good piercing artist, you can start by asking others about their piercings and who did them. Perhaps you have a friend who has a beautifully done piercing and you ask them for a referral to their artist.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6048,"error":null,"output":"B"}
{"question_id":"hellaswag:00009785","item_index":135,"attempt":0,"prompt_hash":"f7272f3ba7a0e8a24f67549a50b7ac59e29bd5f8ce07fd734064fbfbc04b08ff","question":"[header] How to impress your boyfriend [title] Dress to impress. [step] You don't have to be in a ball gown all the time, but do put some thought and effort into what you wear. [substeps] Impress him with cool, flattering outfits even when you're just lounging around at home.\n\nA. Just make sure that you're not wearing something too revealing. He may not notice you're up to something, especially if you're just lounging about the place, so you should take his sense of style into account.\nB. Don't overdo it the first few times. Whether you like wearing black pants or a green button down dress, keep your look simple and elegant.\nC. Wear some interesting accessories and choose items that you think would make you stand out. [title] Don't worry about the right footwear.\nD. Wear his favorite colors or his favorite fragrance to catch both his eye and nose. If you know he's a sucker for a particular outfit in your closet, be sure to wear it on your next big night out.","prompt":"[header] How to impress your boyfriend [title] Dress to impress. [step] You don't have to be in a ball gown all the time, but do put some thought and effort into what you wear. [substeps] Impress him with cool, flattering outfits even when you're just lounging around at home.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6045,"error":null,"output":"D"}
{"question_id":"hellaswag:00009366","item_index":136,"attempt":0,"prompt_hash":"a3e0bdfdd65a17e56f1979fefa2313306d054d36f2461d5a5c33de4b22a9ba9e","question":"[header] How to be talented [title] Revisit your childhood. [step] A good way to figure out where your talents lie is to go back to your childhood and think about what it was you wanted to do as a kid. This is often the time when you had plans that weren't limited by what people tend to think of as \" reality.\n\nA. \" in this age, you can immediately start to determine which means you wish to pursue. [substeps] Look back on your childhood to determine what motivated your talent, why, and how you learned that talent and what stood in your way.\nB. \" if you get frustrated with your ideas of what you want to become, just re-think them. [substeps] Instead of thinking about how you look just before going out and making a list of all of your good qualities, think of all of the good you have done.\nC. \" [substeps] Fear of failure is one of the things that tends to hold you back from achieving or finding your talent. By returning to childhood, you're taking yourself out of that fear of failure or limitations mindset.\nD. \" for instance, maybe you initially wanted to be a broadway singer but later thought you wanted to make a big change. Try to get back in your step-step from childhood.","prompt":"[header] How to be talented [title] Revisit your childhood. [step] A good way to figure out where your talents lie is to go back to your childhood and think about what it was you wanted to do as a kid. This is often the time when you had plans that weren't limited by what people tend to think of as \" reality.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6045,"error":null,"output":"C"}
{"question_id":"hellaswag:00009882","item_index":137,"attempt":0,"prompt_hash":"f64b57d7193917067f2f48b196468a11004a36dcf5c84d341c0bd92e50036176","question":"[header] How to apply face cream [title] Start with a clean face and hands. [step] Wash your face with lukewarm water and a face cleaner suitable for your skin type. Splash your face with cold water and gently pat it dry with a soft towel.\n\nA. [title] Mix the two products with a washcloth and apply it over your face. [step] If you don't have a washcloth or towel, use a warm washcloth or bowl.\nB. [substeps] Pay special attention to your eyes and ears to ensure moisture is preserved. For face creams, use warm water to wash them off.\nC. [title] Consider applying some toner with a cotton ball or cotton pad. [step] The toner will help restore your skin's ph.\nD. [title] Apply a non-irritating cleanser to your face in an inconspicuous place. [step] The cleanser should be cleansing or benzoyl peroxide.","prompt":"[header] How to apply face cream [title] Start with a clean face and hands. [step] Wash your face with lukewarm water and a face cleaner suitable for your skin type. Splash your face with cold water and gently pat it dry with a soft towel.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6044,"error":null,"output":"C"}
{"question_id":"hellaswag:00005020","item_index":138,"attempt":0,"prompt_hash":"c06fc1d7b260433cf4514e60543c5839fa861cb8d66e5b97adcbe1bbdb548e69","question":"[header] How to punish a child for being naughty [title] Set forth clear expectations and consequences. [step] Make sure your child knows exactly what is expected of him/her, and what will happen they break the rules. You can teach your child about the consequences of their actions by explaining the connection between choices and consequences.\n\nA. Be clear about specific punishments and consequence requirements so your child understands that punishment is a punishment rather than an punishment. [substeps] Set a time to discuss the consequences with your child, and give him/her a specific setting to use to avoid misunderstandings.\nB. Starting with simplest consequences can allow your child to make the right decision. [substeps] For example, if your child begs for a toy but asks his parents to take it out of their room, immediately start by saying \" no.\nC. For example, you can connect your child's behavior to consequences by saying things such as : [substeps] \" you are choosing to end your time at the park early by misbehaving. \" \" you lost your turn to play with the toy when you stole it from another child.\nD. This could be a way of reminding them of their consequences when they start to behave in mean ways. [substeps] You may say something such as, \" if your bad behavior does not stop after a certain period of time, you will have to stop the behavior for whatever reason and your punishment will be continued.","prompt":"[header] How to punish a child for being naughty [title] Set forth clear expectations and consequences. [step] Make sure your child knows exactly what is expected of him/her, and what will happen they break the rules. You can teach your child about the consequences of their actions by explaining the connection between choices and consequences.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6046,"error":null,"output":"C"}
{"question_id":"hellaswag:00008992","item_index":139,"attempt":0,"prompt_hash":"eb1f292dd6201c9919a32caced3f6fc9c6d0c0d759b49fd36e7eba6ff8dbd31b","question":"[header] How to dispose of burnt firewood and ash safely [title] Allow the wood and ashes to cool completely in the fireplace. [step] If indoors, let your embers cool down completely with the fire screen closed. Even if it appears the ashes and wood aren't hot, they may remain warm and capable of starting a fire for some time.\n\nA. [substeps] Leaving your embers out on a sunny rooftop, for example, will make it difficult for smoke to escape from the room. Placing several smokers on this rooftop is safer for the building than putting them on a shared porch, which would cause them to burn out.\nB. [substeps] This should only be done if the ash touches a fireplace surface. [title] Provide your kindling at least 15 inches (30 cm) over wood shelf height.\nC. [substeps] Leaving burned wood out in the air creates carbon dioxide, which traps organic compounds. The ashes can then be reused to add gas to your stove or any other flammable surface.\nD. Allow several hours for the embers to cool. They will not radiate any heat when they are completely cold.","prompt":"[header] How to dispose of burnt firewood and ash safely [title] Allow the wood and ashes to cool completely in the fireplace. [step] If indoors, let your embers cool down completely with the fire screen closed. Even if it appears the ashes and wood aren't hot, they may remain warm and capable of starting a fire for some time.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6044,"error":null,"output":"D"}
{"question_id":"hellaswag:00001910","item_index":140,"attempt":0,"prompt_hash":"ea903522e97a860d665d8c55c9dc4823626a69b45dd59da9b83992d8a61c7803","question":"A group of cheerleaders are seen flipping together while a woman speaks to the camera. more people\n\nA. perform tricks on the stage while speaking to the camera.\nB. are seen speaking to the camera that transitions from cheerleaders performing tricks.\nC. speak to the camera and begin dancing while others watch on the side.\nD. cross the beam while the camera captures them, with occasional laughs being pulled in.","prompt":"A group of cheerleaders are seen flipping together while a woman speaks to the camera. more people","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6048,"error":null,"output":"B"}
{"question_id":"hellaswag:00007010","item_index":141,"attempt":0,"prompt_hash":"e0b6735a5d0a25e4552266c9687d843ddc526bc4cca9da2f17ce5e2088a487c3","question":"[header] How to get smokey eyes with makeup [title] Choose your colors. [step] Any color can be used to create a smoky eye, although you will need at least three shades of a similar hue. The classic smoky eye is created with black or gray, but bronze and brown are also commonly used.\n\nA. [substeps] Green eyes look particularly good with a gray and plum smoky eye, while blue eyes pop with gold or copper and brown eyes go great with shades of navy and gray. You should choose three shades of each color: a light, creamy color; a medium base color; and a dark, smoky color.\nB. Black is usually used, but you will need to consider depending on the mood you want to convey. If you want a smoky eye, a black lid is most attractive, and so is black eye liner.\nC. Your choice of colors depends on what face you have to look at. [substeps] Browns with cool blues work great as a base shade for smokey eyes.\nD. [substeps] If you are looking to make smoky eyes on people else, try buying them in neutral shades: ivory, olive, taupe and light beige. If you are going to make these eyes on another person, avoid bright colors, like purple or yellow.","prompt":"[header] How to get smokey eyes with makeup [title] Choose your colors. [step] Any color can be used to create a smoky eye, although you will need at least three shades of a similar hue. The classic smoky eye is created with black or gray, but bronze and brown are also commonly used.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6045,"error":null,"output":"A"}
{"question_id":"hellaswag:00002930","item_index":142,"attempt":0,"prompt_hash":"58707d0783ae515444f2b9091ba82409f5b8694321f27066bd0f30adf698e74b","question":"Woman puts butter in a bowl with sugar, and flour and mix it in a kitchen aid, adding eggs, vanilla and flour, when the mix is homogenous add chocolate chips. the dough\n\nA. is flat and baked in layers, then the woman flour water to steam and make it in flour, then put chocolate chips in a pan and add eggs.\nB. is molded in balls and in a tray to the oven.\nC. is flattened with a spatula, then the woman mix the dough, then they simmer and make a larger plate.\nD. is placed and the sugar is browning and sprinkled in a white plate.","prompt":"Woman puts butter in a bowl with sugar, and flour and mix it in a kitchen aid, adding eggs, vanilla and flour, when the mix is homogenous add chocolate chips. the dough","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6047,"error":null,"output":"B"}
{"question_id":"hellaswag:00004392","item_index":143,"attempt":0,"prompt_hash":"bc9d984dca2c577196f4b6f99b09f2dafc7363a77eebe7881a83f519f4a9b982","question":"[header] How to know if a man is interested in you [title] Look at how he acts towards you vs others. [step] Is he paying way more attention to you than other girls (or even his own friends)? Is he nicer or does he try to touch you more? These may be signs that he's interested in you. [title] Check the body language.\n\nA. [step] He may be acting friendly towards you because he likes your body. If his body language is dismissive of some aspect of your body, his interest may start to decrease.\nB. [step] Many men can communicate better when their partners are close. If you're taller, look for discomfort in your partner's body.\nC. [step] If you see a guy walking towards you with his open body language then he may like you. He may be interested in you but don't take the hint.\nD. [step] Notice how, when, and how often he touches you. A touch to the cheek or neck is a dead giveaway, but a touch to the hand, arm, or lower back can also be a sign that he wants to get closer.","prompt":"[header] How to know if a man is interested in you [title] Look at how he acts towards you vs others. [step] Is he paying way more attention to you than other girls (or even his own friends)? Is he nicer or does he try to touch you more? These may be signs that he's interested in you. [title] Check the body language.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6046,"error":null,"output":"D"}
{"question_id":"hellaswag:00000458","item_index":144,"attempt":0,"prompt_hash":"09d357ad60622b26437c330cf141691a2100c1bc72a15566ffbf6911e504fa63","question":"The man is bent on his knees and suddenly he jumps up to his feet, but keeps his knees bent while he squats low to the ground continuing to talk. the man\n\nA. walked along the rocky beach and jump with the hands on his knees, jumped up in the air and also jump with the arms extended lost.\nB. is still explaining the importance of holding hands with someone who has been on the gymnastics equipment for over 15 years until the video is taken actually.\nC. then takes the racket, pulls the handle again and returns to the previous position in front of him only this time he takes another hit.\nD. then puts his two hands on the ground, and pushes his feet back to go back on his knees and demonstrates one more time how he jumps from his knees to his toes.","prompt":"The man is bent on his knees and suddenly he jumps up to his feet, but keeps his knees bent while he squats low to the ground continuing to talk. the man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6047,"error":null,"output":"D"}
{"question_id":"hellaswag:00007100","item_index":145,"attempt":0,"prompt_hash":"e234e49629908899f896175268972f8a876875905347182546ef80abc33c8b99","question":"[header] How to avoid becoming a fashion victim [title] Resist blindly following trends. [step] While the celebrities might look great in their latest get-up, it's not necessarily a look that will work for you. Remember the low-rise jean fad? If you had a body reminiscent of christina aguilera in the early 2000s, the trend may have worked for you.\n\nA. For most everyone though, it was rather unflattering. [substeps] Give the fashion mags and blogs a break, and resist the urge to buy the latest trendy item if you don't see yourself actually wearing it more than a couple times.\nB. Try to cut it off in some favor, though. By showing confidence when you come across the runway, you can slip past that self-perception and become a more vibrant young lady.\nC. On the contrary, fashion icons often chock down on trendy trends. What if you went for a simple, not revealing look? On the contrary, incorporate fashion everywhere you go for a super chic reason.\nD. Now, trends have fallen into disrepair and you've decided it's time to replace them with style changes and fashion art. Rebuilding your confidence without letting fashion creep in will be quite a challenge.","prompt":"[header] How to avoid becoming a fashion victim [title] Resist blindly following trends. [step] While the celebrities might look great in their latest get-up, it's not necessarily a look that will work for you. Remember the low-rise jean fad? If you had a body reminiscent of christina aguilera in the early 2000s, the trend may have worked for you.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6049,"error":null,"output":"A"}
{"question_id":"hellaswag:00007088","item_index":146,"attempt":0,"prompt_hash":"4478618730b7dc5a9e9b201452a0cf99d856a17e0f3bddd1a1dbd6c339227641","question":"[header] How to handle your baby's stranger anxiety [title] Notice signs of anxiety. [step] Signs of stranger anxiety can begin as early as four months. Some babies may go quiet or look at the stranger with fear.\n\nA. Others may have nervous signals or shake their heads from time to time. You may notice that the stranger is always looking around or doodling on a book.\nB. Others may start talking quietly at night and ignore the baby. [substeps] Babies may have a hard time recognizing even the most seemingly innocuous things.\nC. You can help deal with this by observing how your baby fills her head with fear. [substeps] The child may also start acting out, starting biting her nails or looking at strangers quietly.\nD. Others may become clingy or bury themselves in you. Some children begin to cry, scream, or become fussy.","prompt":"[header] How to handle your baby's stranger anxiety [title] Notice signs of anxiety. [step] Signs of stranger anxiety can begin as early as four months. Some babies may go quiet or look at the stranger with fear.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6046,"error":null,"output":"D"}
{"question_id":"hellaswag:00002255","item_index":147,"attempt":0,"prompt_hash":"5008941f99a6849c143f5c942f29557298a13bc0e1114433dd1bfbb44ac91453","question":"The men then bow before one another as people continue to sweep the arena and lead into the men fighting. the men\n\nA. gather at the outside of the arena as people continue to dance and laugh.\nB. continuously wrestle one another while the ref moves around them and ends with them walking around then off stage.\nC. fight over the other as well as people applauding and the entire crowd clapping.\nD. continue to fight with several people watching on the side but ends with one dying.","prompt":"The men then bow before one another as people continue to sweep the arena and lead into the men fighting. the men","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6045,"error":null,"output":"D"}
{"question_id":"hellaswag:00005219","item_index":148,"attempt":0,"prompt_hash":"16338421388386a758ec18acd91274b28794682b663d87fad1fa28b2a67db977","question":"[header] How to wear a scrunchie [title] Pick a thinner scrunchie for a modern look. [step] Many scrunchies, especially those popular in the' 90's, were famously thick and had a lot of excess fabric bunched around the elastic band. It is no longer fashionable to have a scrunchie dominate your look.\n\nA. Therefore, look for thinner scrunchies without a lot of excess bulk due to fabric. This makes for a more mature, classy look.\nB. [title] Begin with a mesh scrunchie. [step] Mesh scrunchies are the thinnest scrunchie on the market.\nC. Stick with those loose, stretchy scrunchies, and experiment with different shapes and sizes. [title] Choose a roughly 11 \" thickness for your fringe.\nD. Thinner scrunchies are okay for a tighter, more loose-fitting look. [title] Use a colorful scrunchie for a daytime look.","prompt":"[header] How to wear a scrunchie [title] Pick a thinner scrunchie for a modern look. [step] Many scrunchies, especially those popular in the' 90's, were famously thick and had a lot of excess fabric bunched around the elastic band. It is no longer fashionable to have a scrunchie dominate your look.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6045,"error":null,"output":"B"}
{"question_id":"hellaswag:00008864","item_index":149,"attempt":0,"prompt_hash":"2996e4bafa722d5ec01f17a967209cbdfe78fb8393c7c9c47e9e830d472bdc14","question":"[header] How to act like you saw a ghost [title] Start off by describing a dream. [step] This dream should involve a ghost; for example, you might say, \" i had a dream of a boy dying because he was in a car and the gas ran out in the middle of the train crossing. Then the boy was crushed by the train! [title] Elaborate on the story.\n\nA. [step] Your outline outlines the main details of your dream in a simple, unexpected way. If it's your first time telling a story, your outline might include a jumbled up scene or a whole scene.\nB. [step] Say that in a next dream he was talking to you and he was mad/sad or happy and warning you of something. [title] Reveal that you feel someone is watching you.\nC. [step] Start explaining why you know where the ghost is in your dream and why you know how to tell the ghost to go away. Remember, the following are fairy tale stories : [substeps] Dreams involving the swamp, graveyards, etc.\nD. [step] Let your imagination run wild and describe something that occurred in your dream. If you are in a dream where you tell a ghost story about someone scary, try to build your story around it.","prompt":"[header] How to act like you saw a ghost [title] Start off by describing a dream. [step] This dream should involve a ghost; for example, you might say, \" i had a dream of a boy dying because he was in a car and the gas ran out in the middle of the train crossing. Then the boy was crushed by the train! [title] Elaborate on the story.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6045,"error":null,"output":"D"}
{"question_id":"hellaswag:00001281","item_index":150,"attempt":0,"prompt_hash":"9a70f41df0c656663b5557ef4dae4f9333c5b87b3789528a871c5b95cb969146","question":"The man pours water on the knife. The man adds another block on the screen and puts water on the block. the man\n\nA. is pressing buttons on a block.\nB. resumes cutting tiles together, after which the man puts a sponge down and turns off the camera.\nC. walks out of the room.\nD. begins to sharpen the knife again.","prompt":"The man pours water on the knife. The man adds another block on the screen and puts water on the block. the man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6732,"error":null,"output":"D"}
{"question_id":"hellaswag:00004777","item_index":151,"attempt":0,"prompt_hash":"e282574ffb83ed3d046a4468ceadd7f87f39ee0a6b312a8d296d2547fe78fd62","question":"[header] How to sew a basic bag [title] Cut a big square or rectangle out of a nice piece of fabric. [substeps] Depending on the end-use, you may wish to match the fabric to ensure that the bag is strong, somewhat water-resistant, etc. [title] Fold the piece of fabric in half.\n\nA. [step] Have the side of the fabric you don't want to see facing outwards. [title] Sew along the two side edges.\nB. [title] Sew the pieces together as neatly as possible. [step] When doing so, place your fingers or thumb on the fabric at the inside edge for added support.\nC. [title] Get the piece of fabric as square as possible with the square side facing towards the support. [title] Fold this corner over until it meets the other corner.\nD. [title] Fold the long side around so that it is roughly 2-4 centimeter (5-8 in) across. [step] Let the edge slightly flatten once you fold it that direction.","prompt":"[header] How to sew a basic bag [title] Cut a big square or rectangle out of a nice piece of fabric. [substeps] Depending on the end-use, you may wish to match the fabric to ensure that the bag is strong, somewhat water-resistant, etc. [title] Fold the piece of fabric in half.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6732,"error":null,"output":"A"}
{"question_id":"hellaswag:00008053","item_index":152,"attempt":0,"prompt_hash":"d08cd92e0f3b499b763392814eff3cd3b742ba412be55c9451e3603fd25497c5","question":"[header] How to use cleansing milk [title] Pull your hair back. [step] Because you will be leaning forward when you use cleansing milk, you need to secure your hair so it doesn't fall into your face. Pin back your bangs with a clip.\n\nA. Pull your hair into a ponytail with a hair tie. [substeps] If you have short hair, you can hold it back with a headband instead.\nB. [title] Squirt a small amount of cleansing milk into a small bottle. [step] The bottle should fit comfortably but be big enough to contain about a tablespoon of cleanser, although if you have the small bottle you don't need to use that much.\nC. This way you can walk around a bit more and have room to work. [substeps] When you pull back your bangs, you will get a good coverage on your face.\nD. Then, the rest of your hair should be covered. Take a deep breath in and out through your nose, hold it for a few seconds, and then let it go.","prompt":"[header] How to use cleansing milk [title] Pull your hair back. [step] Because you will be leaning forward when you use cleansing milk, you need to secure your hair so it doesn't fall into your face. Pin back your bangs with a clip.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6731,"error":null,"output":"A"}
{"question_id":"hellaswag:00005630","item_index":153,"attempt":0,"prompt_hash":"13c3d6d07105da7d95cb1ee20995e3e6e6eef3ae01ae9b805470eb063f1c92e5","question":"[header] How to prevent canine infectious hepatitis [title] Get your dog vaccinated. [step] The best way to prevent canine infectious hepatitis is to get your dog vaccinated against the disease. This vaccination can be done at the same time as a distemper vaccination, starting when a dog is 9 to 12 weeks old.\n\nA. [substeps] Older dogs with an unknown vaccination status should be given the vaccine followed by a second one in 3-4 weeks. After the initial vaccines are done, it is common practice for a puppy or dog to get a yearly follow up vaccine to provide continuous protection.\nB. It is painless and can be administered by your vet if necessary. [substeps] Hsv is not the same as hepatitis, and can be spread through saliva, sexual contact, or intravenous fluids.\nC. However, if your dog's immune system is weaker, then it may be resistant to doing this vaccine. Vaccination typically takes place 30 to 60 days after a dog has had his shots, preventing the transmission of the virus.\nD. The cdc recommends that the average dog been vaccinated for canine hepatitis does not exceed 9 to 12 weeks and is highly contagious. The cdc recommends getting your dog screened for hepatitis every 6 months.","prompt":"[header] How to prevent canine infectious hepatitis [title] Get your dog vaccinated. [step] The best way to prevent canine infectious hepatitis is to get your dog vaccinated against the disease. This vaccination can be done at the same time as a distemper vaccination, starting when a dog is 9 to 12 weeks old.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6732,"error":null,"output":"A"}
{"question_id":"hellaswag:00004951","item_index":154,"attempt":0,"prompt_hash":"e27d2e9224a1141d93cd063506943f6d73b0e5b473bb8914c21b819ce3325ab6","question":"[header] How to get rid of oily skin fast [title] Prepare your skin. [step] Soak a clean cotton towel in hot water. Wring it out to remove the water and wipe the moistened towel over your skin.\n\nA. Doing this allows the oil and dirt to soak into your skin as effectively as possible. The warm water will also soften your skin, which in turn can help it to heal faster.\nB. Let the drops sit over the spot for up to 1 minute. [substeps] Don't forget to remove the towel after several minutes; this will soften the soap and help the skin lather more efficiently.\nC. Take a cleansing cloth and gently rub in either the liquid or lukewarm water. Another spritz-with soap-over your shoulders and neck is acceptable.\nD. Then, splash your face with lukewarm water to help rinse off any dirt. [substeps] The heat from the towel can help loosen any dirt that's in your pores, making it easier to rinse out.","prompt":"[header] How to get rid of oily skin fast [title] Prepare your skin. [step] Soak a clean cotton towel in hot water. Wring it out to remove the water and wipe the moistened towel over your skin.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6733,"error":null,"output":"D"}
{"question_id":"hellaswag:00004421","item_index":155,"attempt":0,"prompt_hash":"bebb9a5a78838dbb111a2330d49814e52f1bcafa3636af1886b4ce89a6ff4b21","question":"[header] How to make your ex girlfriend want you again [title] Try the no contact rule. [step] Determine a set amount of time where you will no longer interact with your ex. This includes phone calls, text messages, social media, and in person meetings.\n\nA. You are not to initiate contact or respond to her at all. This will give both of you space to heal and work out your feelings.\nB. A few days is a good range if you are unable to talk to your ex again, unless you plan to contact her again. [substeps] Give your friend or family member a call if your ex calls you.\nC. Taking the time to truly acknowledge the situation can be difficult, but it can be a good idea in the beginning. This will give you time to ensure you are not leaving someone on the rebound.\nD. You want to know where she works, where she spends most of her time, and who she hangs out with. Not all relationships work for the same person, and some people may not want to spend time with you more than once during breaks or weekends.","prompt":"[header] How to make your ex girlfriend want you again [title] Try the no contact rule. [step] Determine a set amount of time where you will no longer interact with your ex. This includes phone calls, text messages, social media, and in person meetings.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6615,"error":null,"output":"A"}
{"question_id":"hellaswag:00009399","item_index":156,"attempt":0,"prompt_hash":"c0480d3e6887dc517d0f7fc66dd26438421ca76ffd8328cb7b9e437fddb0b9bc","question":"[header] How to be a gorgeous teen girl [title] Splash your face with warm or hot water in order to open your pores. [step] Use a cleanser that is fit for your skin type and massage it into your face for about a minute targeting areas where acne is located. [title] Rinse your face with the coldest water your face can bear.\n\nA. [title] Pat your face dry and apply an astringent or toner all over your face and give it a second or two to dry. [step] Apply a moisturizer all over your face.\nB. [step] Cold water fills up your pores with great oils. Put in your face one (or two) times a day to get the sulfates flowing out of it.\nC. [step] For a more spf, get yourself a facial mask or a facial scrub with ingredients like avocado, aloe vera juice or almond oil. [title] Gently squeeze your face to try to remove any dirt or oils.\nD. [step] This minimizes the likelihood of irritation. [title] Cover your face with a cotton ball to absorb excess water.","prompt":"[header] How to be a gorgeous teen girl [title] Splash your face with warm or hot water in order to open your pores. [step] Use a cleanser that is fit for your skin type and massage it into your face for about a minute targeting areas where acne is located. [title] Rinse your face with the coldest water your face can bear.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6615,"error":null,"output":"A"}
{"question_id":"hellaswag:00005041","item_index":157,"attempt":0,"prompt_hash":"14c2a7fd3a4be37722b3ac6d97d8722f46928303f985f9076380da99e0fd6697","question":"[header] How to talk to a girl at school (boys ) [title] Stay calm. [step] Try not to overthink the situation. Though you want to talk to this girl very much, remember that she is not perfect and may be as nervous as you.\n\nA. This really is not a good thing, so just relax and let her do all of the talking. Don't try to interrupt her while she is talking.\nB. Girls have emotions and feelings just like you do and may be wondering the right way to approach you as well. Or she may be waiting on you to make the first move! .\nC. But try to become as calm and collected as possible when you talk to this girl. [substeps] Give her your undivided attention instead of worrying about how to react or what you should say next-that way you won't get ruined.\nD. She may come around with small talk at first, but always be prepared. [substeps] For example, if you have nothing to talk about, just talk about something else, like your school, the weather, and your friends.","prompt":"[header] How to talk to a girl at school (boys ) [title] Stay calm. [step] Try not to overthink the situation. Though you want to talk to this girl very much, remember that she is not perfect and may be as nervous as you.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6615,"error":null,"output":"B"}
{"question_id":"hellaswag:00004437","item_index":158,"attempt":0,"prompt_hash":"8e488ac8ee555ec99cd799152cd2e77dced5aa77427287bbbb2763fa5e838b61","question":"[header] How to hide an erection [title] Wear tighter underwear. [step] Instead of wearing loose fitting boxers that make an erection visible and obvious, consider wearing tighter underwear like briefs or boxer briefs that can hold down your erection and make it less obvious. [substeps] Avoid wearing soft silk underwear because they can rub against your penis and cause you to have an erection.\n\nA. [title] Clamp the hand on your penis firmly to keep your erection erect. [step] The easiest way to cope with an erection is to clamp your hand tightly on your penis.\nB. Wear boxers that cannot slide off if you are not wearing boxer briefs. Resist wearing tight fitting boxers, as having tighter underwear that shrinks can make your erection worse.\nC. Wearing heavy underwear can also make the erection easier to hide. You should see an emergency medical professional to see if you are struggling with uncomfortable clothing for more serious reasons.\nD. [title] Cover your crotch area with a large object. [step] You can cover up your erection with a book, blanket, laptop, backpack, or anything else that can go on your lap.","prompt":"[header] How to hide an erection [title] Wear tighter underwear. [step] Instead of wearing loose fitting boxers that make an erection visible and obvious, consider wearing tighter underwear like briefs or boxer briefs that can hold down your erection and make it less obvious. [substeps] Avoid wearing soft silk underwear because they can rub against your penis and cause you to have an erection.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6615,"error":null,"output":"D"}
{"question_id":"hellaswag:00002879","item_index":159,"attempt":0,"prompt_hash":"1cb69ce9c4649197b9748c149e778b62f0514071c0d56c04e650166b74dbdaed","question":"A boy in an orange shirt is playing a video game. the scene\n\nA. switches to show the players playing indoor soccer.\nB. changes to the game itself.\nC. cuts to the boys playing the game on the swings outside.\nD. changes to safety features.","prompt":"A boy in an orange shirt is playing a video game. the scene","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6613,"error":null,"output":"B"}
{"question_id":"hellaswag:00001981","item_index":160,"attempt":0,"prompt_hash":"10691e94b78b52b99e90ee6db46b6a33dc529f3e727d8b8530bbe8380b6dfd88","question":"A still shot picture of a body of water appears and yellow words appear on the screen and read \"american disk\" and include the website. a brunette woman\n\nA. around 30 is sitting an interviewer walking in front of her as she speaks and then instructions appear and reads \" moist this sponge sponge is used in caring for fish wildlife \".\nB. is cleaning her mouth with a toothbrush as words are shown in the middle of the screen.\nC. is now speaking next to the water and she's smiling the whole time.\nD. , while she is standing and wearing a grey dress is shown performing stomach crunches.","prompt":"A still shot picture of a body of water appears and yellow words appear on the screen and read \"american disk\" and include the website. a brunette woman","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6097,"error":null,"output":"B"}
{"question_id":"hellaswag:00002545","item_index":161,"attempt":0,"prompt_hash":"b3a263580686ccaec922f07261eeba63b4f34c6073f32de4da3f65d729b9cc07","question":"He does a hand stand and then does a front flip as he swings on the bars. He does another hand stand and a front flip as he twists his body. he\n\nA. does a couple of hand springs as he performs front flips.\nB. continues swinging on the bars till he jumps off the bars and lands on his feet.\nC. dismounts from the bars and tosses his pick in the bucket of water.\nD. is done and walks across the mat.","prompt":"He does a hand stand and then does a front flip as he swings on the bars. He does another hand stand and a front flip as he twists his body. he","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6097,"error":null,"output":"B"}
{"question_id":"hellaswag:00003264","item_index":162,"attempt":0,"prompt_hash":"feea16372e60ae1ebf5e1eee9c4aaea465fe88f345c37f73aa10b35bdda8420c","question":"[header] How to make stuffed meatballs [title] Preheat the oven and cut the cheese. [step] Turn on the oven to 350 degrees f (180 c). Take one 8-ounce (226 g) block of mozzarella cheese and use a sharp knife to cut it into 1/2-inch (12 mm) cubes.\n\nA. You should get about 20 small cubes of cheese. [substeps] You could also use pepper jack, cheddar, or monterey jack cheese.\nB. Lay one slice of mozzarella cheese on a sheet of paper. Make sure the sticks of cheese are uniformly placed so the end with 1/2 inch (12 mm) is above the cheese.\nC. Set the cubes aside while you make the rest of the filling. [substeps] If you prefer to make stuffed meatballs using multiple meatballs instead of just one, you can dice a sheet of cheese and place it in the middle of a baking sheet.\nD. [substeps] Use parmesan cheese if you want a richer sauce instead of mozzarella cheese. If you want to save the cheese for later, you can use it right now.","prompt":"[header] How to make stuffed meatballs [title] Preheat the oven and cut the cheese. [step] Turn on the oven to 350 degrees f (180 c). Take one 8-ounce (226 g) block of mozzarella cheese and use a sharp knife to cut it into 1/2-inch (12 mm) cubes.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6090,"error":null,"output":"A"}
{"question_id":"hellaswag:00005846","item_index":163,"attempt":0,"prompt_hash":"14d2cf83b000d2f84a342038a595a96f5d1ee14388bfed1f04232fdf348d4165","question":"[header] How to set up a hissing cockroach house [title] Find a decent size tank depending on how many cockroaches you have. [step] Cockroaches will eat anything, they are omnivorous, this means they eat both fruit and meat. Cockroach's love bananas and fish food.\n\nA. Some try to sell them to non-fish buyers, but this is not recommended. If your cockroaches eat very mushy stuff they will break down and die.\nB. They can consume any submerged creature unless it is fossilized, this is considered a pet. [title] Decide if you wish to keep or not, you can either put it in a well-ventilated area, or put it back where the roaches are.\nC. Some also have bell pepper and celery. [title] Place your tank in a dark and quiet area, where the cockroaches will get bored not because of noise but if they need proper ventilation.\nD. They will appreciate fresh veg and fruit every so often and will also eat dry dog food. Use a small little bowl or a jam jar lid as a sort of plate for their food.","prompt":"[header] How to set up a hissing cockroach house [title] Find a decent size tank depending on how many cockroaches you have. [step] Cockroaches will eat anything, they are omnivorous, this means they eat both fruit and meat. Cockroach's love bananas and fish food.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6090,"error":null,"output":"D"}
{"question_id":"hellaswag:00009912","item_index":164,"attempt":0,"prompt_hash":"0c795d025494e5d657f783020e3495621d5ad1d8dbbc361650cfd47da20f28d2","question":"[header] How to do abdominal exercises in yoga [title] Do the lotus pose to start things off. [step] This is a simple pose that relaxes the mind and prepares it for a full and extensive workout. The pose helps strengthen and stimulate the muscles around your legs and knees, while focusing yourself for the workout ahead.\n\nA. [substeps] This basic pose combines poses of the lotus and the backbend pose. Stand with your feet slightly wider than shoulder width.\nB. [substeps] Start out lying on your back on a yoga mat, with your knees bent and your feet together. Turn the bottom of your torso toward the ceiling and keep your feet firmly on the ground.\nC. [title] Begin with the four legs splayed before you. [step] Cross your right foot in front of your left.\nD. Stay in lotus for as long as you wish, focusing on becoming aware of your breathing. The breathing rhythm you achieve in this opening position will set the tone for all positions ahead.","prompt":"[header] How to do abdominal exercises in yoga [title] Do the lotus pose to start things off. [step] This is a simple pose that relaxes the mind and prepares it for a full and extensive workout. The pose helps strengthen and stimulate the muscles around your legs and knees, while focusing yourself for the workout ahead.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6091,"error":null,"output":"B"}
{"question_id":"hellaswag:00001802","item_index":165,"attempt":0,"prompt_hash":"5002a6109dd6c5cbc47cc3b361f141884d2345dea62093236127b31c03897a8c","question":"Men is standing in stage playing frums. Blonde woman wearnig a white dress is playing violin walking around the stage. behind the girl\n\nA. people are playing different instruments.\nB. is a crowd gathered in the cathedral.\nC. is a curtain o people are playing violin.\nD. men is stretching in stage.","prompt":"Men is standing in stage playing frums. Blonde woman wearnig a white dress is playing violin walking around the stage. behind the girl","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6094,"error":null,"output":"A"}
{"question_id":"hellaswag:00003865","item_index":166,"attempt":0,"prompt_hash":"1b3a6553115b23bc068ea8054d31dc99d5a247b00577e7be7f7ead7e7a49d78e","question":"[header] How to become a wedding officiant [title] Check your state's laws. [step] A wedding officiant is fulfilling the legal duty of certifying a marriage license so it's important to understand the law in your state. [substeps] Visit your state's secretary of state website to see which laws apply.\n\nA. Although you don't need to do this to become a wedding officiant, you should do it according to state laws. If you live in an area where such laws apply, keep yourself informed of those laws and methods in your state.\nB. There are no regulations that require a marriage officiant to perform a wedding in your state. In most states your officiant must be at least 18 and older than you.\nC. If you'd like to prove officiant duties in your state, check with your local attorney. If you've never officiated a wedding in your state before, do so before completing the marriage license application (with irs form 5536).\nD. Some states are very lenient with who can become an officiant, but others require officiants to be a practicing minister with a congregation. Some states will impose a fine for any marriages officiated illegally.","prompt":"[header] How to become a wedding officiant [title] Check your state's laws. [step] A wedding officiant is fulfilling the legal duty of certifying a marriage license so it's important to understand the law in your state. [substeps] Visit your state's secretary of state website to see which laws apply.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6090,"error":null,"output":"D"}
{"question_id":"hellaswag:00008691","item_index":167,"attempt":0,"prompt_hash":"8e59b2de4eaf98a65c35ad5a3257a0c9e5a00b481529109b8e3b4916488f52c7","question":"[header] How to sort on google sheets on android [title] Open the google sheets app on your android. [step] The sheets app looks like a white spreadsheet table on a green document icon. You can find it on your apps menu.\n\nA. [title] Tap the file you want to sort. [step] The checkmarks on the page will indicate that the file is being sorted.\nB. [substeps] If you don't have sheets on your android phone, you can download them for free from the google sheets app on a new device. [title] Tap the blank boxes on the spreadsheet screen.\nC. [substeps] If you are running a windows or mac computer, you'll find it in the app drawer. If you are running a mac, but you have just created a document, tap the file menu in the upper right-hand corner.\nD. [title] Tap the file you want to edit. [step] Find the spreadsheet you want to edit on your saved files list, and open it.","prompt":"[header] How to sort on google sheets on android [title] Open the google sheets app on your android. [step] The sheets app looks like a white spreadsheet table on a green document icon. You can find it on your apps menu.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6092,"error":null,"output":"D"}
{"question_id":"hellaswag:00009404","item_index":168,"attempt":0,"prompt_hash":"90f98fd29a0e5d20e16bd4061d7ea47714533ce7b4ed25b4960576915d412a80","question":"[header] How to maintain leather shoes [title] Remove dirt with a soft cloth. [step] Rub off as much dirt and grime as you can with a clean, dry cloth. Try to scrape off any loose mud, for instance, and wipe away any excess salt.\n\nA. [title] Fill a container with white vinegar or lemon juice. [step] Thin, still-clean, white vinegar has antibacterial properties that make applying the vinegar more effective.\nB. [substeps] If you are working with other materials, like a paper towel, this method may take some time to complete. If necessary, have a friend help you finish the job.\nC. [title] Apply a cleaner with a soft brush. [step] If the cleaner comes with an applicator top, use it to rub the cleaner into the shoe.\nD. Continue cleaning the shoes until all the soap has been completely washed out. [substeps] Be very careful not to scrub your shoes too hard, as doing so could cause the shoes to crack or warp in their wear and tear.","prompt":"[header] How to maintain leather shoes [title] Remove dirt with a soft cloth. [step] Rub off as much dirt and grime as you can with a clean, dry cloth. Try to scrape off any loose mud, for instance, and wipe away any excess salt.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6090,"error":null,"output":"C"}
{"question_id":"hellaswag:00001228","item_index":169,"attempt":0,"prompt_hash":"86c3e9509442f053a6ba42cd46e34fd692b3bebc74eae5e859fff473801dde6d","question":"A man named bryan mcbride is standing and then begins his high jump where he clears it, lands and vigorously cheers as he runs off. the stats for the men's high jump\n\nA. are base and beam.\nB. are shown on the screen and he begins his high jumps and practices and then runs off and his 10lb is shown again.\nC. he jumped are shown.\nD. appear and it shows that he is now listed as #1 with the best score.","prompt":"A man named bryan mcbride is standing and then begins his high jump where he clears it, lands and vigorously cheers as he runs off. the stats for the men's high jump","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6093,"error":null,"output":"D"}
{"question_id":"hellaswag:00005840","item_index":170,"attempt":0,"prompt_hash":"b1a8662934a24c443d19519e46a85e81b8c4d4fe002be1db017173c0ce3dfbfb","question":"[header] How to add friends on snapchat [title] Find out your friend's username. [step] The main way people add each other in snapchat is by trading usernames. This allows you to add people that aren't in your phone's contact list.\n\nA. Each time a person will be added to your phone's contacts list, you save your current chatting list. [substeps] Snapchat has a feature to fit most smartphones-whatsapp, safari, or similar.\nB. Since snapchat is a long-distance connection between people, you can use it to your advantage and get friends with similar names. [substeps] Snapchat is often used on facebook and other social media platforms.\nC. There is no way to add someone to snapchat using an email address or real name. You'll need to ask the person what their snapchat username is.\nD. Many people swap their usernames with a different user in order to easily get more followers. This doesn't give you enough available space to add two people.","prompt":"[header] How to add friends on snapchat [title] Find out your friend's username. [step] The main way people add each other in snapchat is by trading usernames. This allows you to add people that aren't in your phone's contact list.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6089,"error":null,"output":"C"}
{"question_id":"hellaswag:00004326","item_index":171,"attempt":0,"prompt_hash":"2f8d002c95b48728a25e44b4aeba8c5d5aecd226f839981c3c0852daff8ffb01","question":"[header] How to care for dry curly hair [title] Apply a deep conditioner just before you get in the shower. [step] Work a liberal amount of deep conditioner into your hair section by section, pulling apart your curls and coating each strand. The longer and thicker your hair and the tighter your curls, the more conditioner you will need.\n\nA. [title] Apply a shower cap to help lock in humidity and keep your hair from becoming clogged, whereas keeping the moisture out will help you sleep soundly. [step] In between showers, apply a deep conditioner with spf 15 or higher if your hair is very damp.\nB. [substeps] You can keep your hair very tight or loose by using a towel. [title] Place your curls under cool water using a brush.\nC. Tuck it up under a shower cap and hop in the shower, washing your body first. [substeps] Once you've kept the cap on for about 10 minutes, remove it and wash your hair as usual.\nD. [substeps] You can have a deep conditioner made at home, or opt for a hair supply store. [title] Rinse the conditioner out with cool water.","prompt":"[header] How to care for dry curly hair [title] Apply a deep conditioner just before you get in the shower. [step] Work a liberal amount of deep conditioner into your hair section by section, pulling apart your curls and coating each strand. The longer and thicker your hair and the tighter your curls, the more conditioner you will need.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6090,"error":null,"output":"C"}
{"question_id":"hellaswag:00000131","item_index":172,"attempt":0,"prompt_hash":"3927ccd70809e0f0956536e0e6a42a221d446f75392d5112ca7ed27b5065ef15","question":"A runner is running on a track in front of a crowd of people. a man\n\nA. is shown talking to the camera.\nB. is not running when he jumps into a sand pit.\nC. gives a brief introduction before continuing and competing against the man in black.\nD. is throwing a ball for his dog to catch.","prompt":"A runner is running on a track in front of a crowd of people. a man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6091,"error":null,"output":"A"}
{"question_id":"hellaswag:00008948","item_index":173,"attempt":0,"prompt_hash":"8f3ea0e8d2e29c2f9037df056919de5e5085b9dbbb760d306628163a111ce46c","question":"[header] How to show affection to your wife [title] Give her your full attention when she talks. [step] If you tend to get distracted by your phone or computer while talking, turn them off or ask to move the conversation to a different location. You can show her that you are really paying attention by getting rid of all distractions, making eye contact, and responding at the appropriate moments.\n\nA. [substeps] Instead of constantly staring at your phone or computer, try putting your phone in your pocket. This way, she might be less likely to get lost in her head and make a rapid change in the direction you are looking.\nB. [substeps] Avoid whispering or picking up old books or magazines. Take cues from your wife's specific communication styles.\nC. [substeps] During larger meals together make eye contact with her, talk about something you have already shared, then forget about the conversation altogether. If she makes eye contact while talking, then it means that she wants to talk.\nD. [title] Take a few minutes to check in every day. [step] Even taking a few minutes each day to talk with your wife can strengthen your relationship.","prompt":"[header] How to show affection to your wife [title] Give her your full attention when she talks. [step] If you tend to get distracted by your phone or computer while talking, turn them off or ask to move the conversation to a different location. You can show her that you are really paying attention by getting rid of all distractions, making eye contact, and responding at the appropriate moments.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6091,"error":null,"output":"D"}
{"question_id":"hellaswag:00001282","item_index":174,"attempt":0,"prompt_hash":"9bd2c3666c87e37315486c62a45eacc312803441fc058f18850bbfd07a2dfea4","question":"The man adds another block on the screen and puts water on the block. The man begins to sharpen the knife again. the man\n\nA. finishes finished sharpening the knife.\nB. points to the board then up to the board and continues to sharpen the knife.\nC. stops and shows the knife.\nD. then puts salt on the board and adds the different block in a circle.","prompt":"The man adds another block on the screen and puts water on the block. The man begins to sharpen the knife again. the man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6091,"error":null,"output":"C"}
{"question_id":"hellaswag:00006908","item_index":175,"attempt":0,"prompt_hash":"df147cf73dd445a89ca9c05da5868c21318c0c3269fd761d1e0987bd43988a40","question":"[header] How to throw a science fiction party [title] Make sci-fi invitations. [step] Make space-like invitations with star wars creatures, tin foil cut-outs, ufos, aliens, and anything else sci-fi you can think of. Send them out at least a week ahead of time so all your guests are sure to make it.\n\nA. [substeps] If you send invites yourself, make sure your guests know there will be some sort of throw-in at the party. [title] Send out pokémon and ravaras at least a couple of weeks before the event.\nB. [substeps] You can even add star wars character pics if you like. Make sure when you make invitations that others follow them.\nC. Try planning the order after the party (s) to have two already arrived by the time everyone leaves. Keep being and making and looking for inviting invitations that your guests can return to the party on an after.\nD. Invitations should include your contact information for rsvp, time, date, and place of party, what guests should bring, and any other details. [substeps] You can choose a specific tv show to base invitations on.","prompt":"[header] How to throw a science fiction party [title] Make sci-fi invitations. [step] Make space-like invitations with star wars creatures, tin foil cut-outs, ufos, aliens, and anything else sci-fi you can think of. Send them out at least a week ahead of time so all your guests are sure to make it.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6088,"error":null,"output":"D"}
{"question_id":"hellaswag:00000409","item_index":176,"attempt":0,"prompt_hash":"24ec710ed486102a8c0f36a763581ed12f18d78381742e0eb1861529147b32cd","question":"A group of boys are on the sidewalk of a street. They are bouncing around on stilts. they\n\nA. continue talking as they bounce and walk.\nB. have wrapping paper in their hands.\nC. hit hoops each other as they walk.\nD. bounce and flip over a wobbly bar onto a mat.","prompt":"A group of boys are on the sidewalk of a street. They are bouncing around on stilts. they","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6091,"error":null,"output":"B"}
{"question_id":"hellaswag:00005364","item_index":177,"attempt":0,"prompt_hash":"ff46af5b1c61e4a393ab9a06756b785d9304650daf458926f37bd3825b6d595a","question":"[header] How to choose moisturizer for oily skin [title] Rule out problematic products. [step] Don't assume you have naturally oily skin just because it appears shinier than you would like. You might just be using the wrong product.\n\nA. Search for products with natural ingredients in them that irritate oily skin. [substeps] Avoid products that have harsh chemicals and that use salicylic acid for the base to help moisturize your skin.\nB. [substeps] It's possible that the moisturizer you are using is too heavy. When you use a product that is too rich for your skin, your pores can't absorb it.\nC. [title] Use a basic sunscreen over other moisturizing products. [step] A moisturizing product is designed to smooth out the outer layer of your skin, making it look smooth and blemish-free.\nD. Instead, you may have found just the right product that is actually causing your skin problem. Read what other people say about your skin to see if you are aware of any other problem.","prompt":"[header] How to choose moisturizer for oily skin [title] Rule out problematic products. [step] Don't assume you have naturally oily skin just because it appears shinier than you would like. You might just be using the wrong product.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6090,"error":null,"output":"B"}
{"question_id":"hellaswag:00009980","item_index":178,"attempt":0,"prompt_hash":"ce508b5c393dd671c04e897e90f2c8c349e67cb6db388f274a2965b2a9bee0a5","question":"[header] How to choose the right office desk [title] A lamp is just not a source of light on your reading table. [step] It's more of a functional accessory in a room. And just like with any other accessory, there are a lot of choices or varieties of desk lamps to choose from.\n\nA. Here we are going to explore how to select the right desk lamps, as well as what considerations you should factor before the actual process. [title] Factors to consider when selecting a desk lamp [title] Size [title] This is where most people go wrong, when it comes to selecting lamps for their desks.\nB. Examples include a bench lamp (including a' loveseats' lamp) or no lamp at all. The most popular lamps are the ones with diamond and orange lenses.\nC. [title] If your office is very light at the end of the day use a window unit. [step] Or, if you're a busy person and it's just a bedroom, set it near the windows and from out of the way a little bit of light can bleed out from the interior.\nD. Some good lamps to consider are : [substeps] Bring your own lamps with you wherever you go that you buy very regularly. Think of the golden shades that go around with a lamp.","prompt":"[header] How to choose the right office desk [title] A lamp is just not a source of light on your reading table. [step] It's more of a functional accessory in a room. And just like with any other accessory, there are a lot of choices or varieties of desk lamps to choose from.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6091,"error":null,"output":"A"}
{"question_id":"hellaswag:00004156","item_index":179,"attempt":0,"prompt_hash":"4b71b24a9ec7ff8f413e1b454f4dcc68506cc7ffb28a1c9071d5af1749b7f255","question":"[header] How to hammer copper [title] Purchase a copper sheet appropriate to the specifications of your job. [substeps] Get approximately 2 feet (about 61 cm) more copper sheeting than you need for the job. Allow 1 foot (about 30. 5 cm) on each side.\n\nA. Only purchase tools marked custom steel for the job. The iron plate will help greatly if you need to replace the plate of your choice.\nB. Cut the sheets as soon as you arrive at the job site. [title] Get a surface on which to hammer galvanized sheets.\nC. Purchase 2 inches (5.08 cm) pieces and use a calculator to find out how much copper will be needed for the job. Multiply the original amount from your allowance for the job to get the copper sheet.\nD. It will shrink with extensive hammering, and you'll need the excess on the ends for hammering in nails. [title] Choose a soft temper of copper sheet to allow you to hammer copper more easily.","prompt":"[header] How to hammer copper [title] Purchase a copper sheet appropriate to the specifications of your job. [substeps] Get approximately 2 feet (about 61 cm) more copper sheeting than you need for the job. Allow 1 foot (about 30. 5 cm) on each side.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6090,"error":null,"output":"D"}
{"question_id":"hellaswag:00007233","item_index":180,"attempt":0,"prompt_hash":"e56f9bc643f1639628b515d919b29d7c0a034a6fa4a01b2a7e69d97c051a6a6c","question":"[header] How to have perfect and gorgeous skin in less than 15 minutes [title] Use a primer. [step] Using a primer is one of the most important steps in creating great looking skin. It is also one that is commonly skipped.\n\nA. By using primer, you will have a more even color and can spend more time priming your skin, which means you won't have to worry about pricing again and will be able to finish prizing as you go. [substeps] Primer is great to use because primer stands out from the rest.\nB. Plus it comes with something to grab onto. Also, primer is required to help keep you from getting any oily coverage! [substeps] If you have oily skin, using a primer will not provide you with all the coverage you need.\nC. Primer is a product that goes on first when applying your make-up. Think of it as the best foundation for your foundation! Primers create a palette that is bright and smooth, which will make the rest of your make-up look flawless.\nD. Even a primer is important for your skincare. [substeps] After primer sets, apply a primer that is designed to give you a beautiful glow.","prompt":"[header] How to have perfect and gorgeous skin in less than 15 minutes [title] Use a primer. [step] Using a primer is one of the most important steps in creating great looking skin. It is also one that is commonly skipped.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6089,"error":null,"output":"C"}
{"question_id":"hellaswag:00001158","item_index":181,"attempt":0,"prompt_hash":"fe3ec5dd34c6c5977b04d4bc97432e6f3c78645f1d26a6a8b41148b3e69c8047","question":"The man covers the entire chest from the very top to the bottom with the spray varnish which is coming from a long hose attached to a metal pump. the man\n\nA. plays the music cutting around different alifes of the product.\nB. uses the hose on his legs and hands taking off the hose from the hose that is attached to the pump.\nC. finishes spraying the chest and hangs the nozzle up and walks away as the camera focuses on the finished product.\nD. continues plastering the chest and the adapting metal pump and then grabs another container of the paint and begins to use it to paint a height mirror them and use it to scrub excess paint off of it.","prompt":"The man covers the entire chest from the very top to the bottom with the spray varnish which is coming from a long hose attached to a metal pump. the man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6089,"error":null,"output":"C"}
{"question_id":"hellaswag:00001898","item_index":182,"attempt":0,"prompt_hash":"61be5c22480156e38c933059c61d8e872560efec27dfb7957fead7ff1882de4c","question":"Out on cross lake a man is walking dragging his equipment behind him. he\n\nA. is going to take another man down in the water.\nB. is standing and talking near a bottle of propane and ice cream.\nC. is on the lake in a canoe moving fast heavier water water.\nD. finds a little hole in the ice and sets up to fish, moving around here and there before finally settling in his chair.","prompt":"Out on cross lake a man is walking dragging his equipment behind him. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5520,"error":null,"output":"D"}
{"question_id":"hellaswag:00008065","item_index":183,"attempt":0,"prompt_hash":"6fddc27dbc3ab9976d65cfa8a8b18c81e2caa4c9250f4c40b86f67c61df9ecc4","question":"[header] How to prepare for the adoption home study process [title] Attend an information session. [step] Many adoption agencies will provide an orientation you can attend if you are interested in adoption. The session usually is free, and you are not obligated to work with the agency if you attend.\n\nA. This can help you get an idea of their methodology and provide confidence that you understand their methodology and methods. [substeps] When scheduling your sessions, do so in person rather than by email, so you can work with the agency while also attending the meeting.\nB. Attending an information session is a good way to learn about the agency and the adoption process. [substeps] You can contact nearby adoption agencies and ask if they hold information sessions.\nC. The program takes about 90 minutes to complete and will provide a basic background and assessment. You will also learn important medical skills, be prepared to undergo testing, and demonstrate that you are a good candidate for adoption.\nD. A more thorough information session may be provided by some of the agencies that market a dna match. [substeps] The prospective adoptive parents may be asked questions about the prospective adopter, his or her individual medical history and physical characteristics, and the adoption process.","prompt":"[header] How to prepare for the adoption home study process [title] Attend an information session. [step] Many adoption agencies will provide an orientation you can attend if you are interested in adoption. The session usually is free, and you are not obligated to work with the agency if you attend.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5518,"error":null,"output":"B"}
{"question_id":"hellaswag:00009792","item_index":184,"attempt":0,"prompt_hash":"106e3b7d4ae97e9bb88d31524a47f39af4682b79fe3bc79437b276d1bccf0a09","question":"[header] How to increase breast size with a bra [title] Check your bra size. [step] Wearing a bra that is too big will not make your breasts look larger. That is why it is important to wear the right size bra for you.\n\nA. You can visit a lingerie shop and have them measure you to help you fit a bra to your size, or measure yourself before you shop for a new bra to ensure that you get a good fit. [substeps] Use a flexible tape measure, like one found in a sewing kit.\nB. This is why it is important to keep your breasts under protection when in the shower and only wear bras that are larger than your breast size. [substeps] If you are not wearing a bra, try wearing something that is a little bigger.\nC. For a girl, a bra with a support strap will be easier for her, because most women are unable to pull through bra straps and bras that are too small will not be able to support breasts from side-to-side. Many bras have even been created that cover the breast side, and can be sent to other women in the world to make them look bigger.\nD. Choose a color that is flattering to your breast type and specific event, in addition to those that make you uncomfortable. Look for sports bras made from natural material, such as spandex or lycra, as this is a more breathable bra.","prompt":"[header] How to increase breast size with a bra [title] Check your bra size. [step] Wearing a bra that is too big will not make your breasts look larger. That is why it is important to wear the right size bra for you.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5518,"error":null,"output":"A"}
{"question_id":"hellaswag:00000806","item_index":185,"attempt":0,"prompt_hash":"34b5f05a2eb199aedb61ccf6c36032c773b353a1e7ebb40adce78d4880291a8f","question":". a bald man and a woman wearing handkerchief in her hair\n\nA. are practicing blow drying her hair.\nB. are outside of an aerobics class.\nC. stand and wave next to a customized bus.\nD. are sitting in front of a woman doing a tutorial on stain removal techniques.","prompt":". a bald man and a woman wearing handkerchief in her hair","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5517,"error":null,"output":"D"}
{"question_id":"hellaswag:00001087","item_index":186,"attempt":0,"prompt_hash":"87f8298e14a05d125ba032f9d28412b7654dae0445f98df15ac9448b10e7b4cc","question":"This man is standing with his hands on his hips and he explains what exercises he will be doing throughout the video. first he\n\nA. shows viewers how to properly do a crunch and his feet are apart, arms on his chest, and each time he goes up the elbows touch his legs.\nB. adjusts the shelf to his right so that it is more comfortable and he can use it to balance on.\nC. puts some man stilts on and he clips the stilts to his legs.\nD. shows the correct form of the fitness machine, where you can begin by walking down the white line with ankles and neck rests facing upwards.","prompt":"This man is standing with his hands on his hips and he explains what exercises he will be doing throughout the video. first he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5516,"error":null,"output":"A"}
{"question_id":"hellaswag:00005403","item_index":187,"attempt":0,"prompt_hash":"3da85848b38460aec452badd136063804e2f5c1b980878df90ff03c145ac6ce4","question":"[header] How to diagnose autism early [title] Know when signs begin appearing. [step] Some signs may begin to emerge as early as 6 to 12 months. Specialists tend to hold off giving a definite diagnosis until approximately 18 months.\n\nA. Signs can also start as early as 6 weeks after a diagnosis. [substeps] Symptoms may begin as rarely as three months after a diagnosis.\nB. This is due to the patterns of autistic traits in early development. Some of the signs may fade in, and fade back out again.\nC. [substeps] Severe cases typically occur within these months. If you are diagnosed early, symptoms lasting more than a few weeks may begin to have lasting effects on your life.\nD. The early warning signs disappear after this time. [substeps] Symptoms will arise about an hour after onset.","prompt":"[header] How to diagnose autism early [title] Know when signs begin appearing. [step] Some signs may begin to emerge as early as 6 to 12 months. Specialists tend to hold off giving a definite diagnosis until approximately 18 months.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5698,"error":null,"output":"B"}
{"question_id":"hellaswag:00009078","item_index":188,"attempt":0,"prompt_hash":"b0f5b8e593d7276f895b06593ba10684fe76126fd6374f5cabeeb85d01697494","question":"[header] How to write a style guide [title] Check out other style guidelines in your professional area. [step] The area you or your organization writes in likely already has a preferred style guide. If you want to create your own, consider starting with the preferred guide.\n\nA. Consult your organization's preferred style guide before starting with your company's style guide and determine if the style guide you write is true or false. [title] Try something you've never done before.\nB. It can give you an idea of what you want to include in your own. [substeps] You don't have to write a whole new guide if another one covers the bulk of what you want.\nC. However, if you do not feel comfortable designing your own style guide, consider printing it and downloading a template online. [substeps] If you have access to word processing services or a computer with larger images, you may want to get a home style guide.\nD. Look through the guidelines and identify what is appropriate based on your topic (s) and motivations for writing the style guide. [substeps] For example, consider starting with a style guide that gives a brief introduction to characters, relevant thoughts, and processes.","prompt":"[header] How to write a style guide [title] Check out other style guidelines in your professional area. [step] The area you or your organization writes in likely already has a preferred style guide. If you want to create your own, consider starting with the preferred guide.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5697,"error":null,"output":"B"}
{"question_id":"hellaswag:00004501","item_index":189,"attempt":0,"prompt_hash":"01a2deebc25853a2077a858917e9d439c798ce2aa4a1215f67e5b2aefb3d1ea1","question":"[header] How to create an eco friendly nursery [title] Purchase an organic baby crib. [substeps] Consider purchasing a crib made with recycled wood or certified natural wood to be sure your baby's crib is not painted with a harmful varnish. Furthermore, cribs fashioned from natural or recycled woods benefit our environment and allow the consumer to obtain a quality product.\n\nA. Shop early on in your pregnancy to allow enough time for the crib to be constructed and shipped. [title] Note how your crib mattress is manufactured.\nB. There are many natural wood nursery kits that you can purchase cheaply from many pet stores, organic bakeries, hospitals, and clinics. They allow the product to be manufactured and disassembled without requiring a warranty.\nC. Check your crib's material and color scheme before buying a baby crib. [title] Choose a crib material that is appropriate for your baby's feeding or eating schedule.\nD. In addition, the nursery itself is a good choice because soil is renewable. Unfortunately, fertilizers need to be removed from the soil to further protect your baby's environment.","prompt":"[header] How to create an eco friendly nursery [title] Purchase an organic baby crib. [substeps] Consider purchasing a crib made with recycled wood or certified natural wood to be sure your baby's crib is not painted with a harmful varnish. Furthermore, cribs fashioned from natural or recycled woods benefit our environment and allow the consumer to obtain a quality product.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5698,"error":null,"output":"A"}
{"question_id":"hellaswag:00002256","item_index":190,"attempt":0,"prompt_hash":"a3130526f332d46396823df23517f29c80362d20ddc5d47e1aa6339dd9626abb","question":"A lady stands in a dance studio. the lady\n\nA. paints the toes of a shoe.\nB. demonstrates how to use a portable dance partner.\nC. picks up a baton and spins.\nD. performs a ballet dance.","prompt":"A lady stands in a dance studio. the lady","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5697,"error":null,"output":"D"}
{"question_id":"hellaswag:00007115","item_index":191,"attempt":0,"prompt_hash":"d5f23908fa89331b18218e702473f200ea66000aafd0ae025f878d3b80dab435","question":"[header] How to benefit from interpersonal therapy [title] Talk to your doctor. [step] The decision to use interpersonal therapy should be discussed with your doctor. If you have depression or a similar condition, you may decide to try interpersonal therapy prior to medication, or you may decide to combine medication with the therapy.\n\nA. Once you have discussed your concerns, you can decide which therapy approach is right for you. [substeps] Determine if you want to involve a doctor or a therapist in your therapy.\nB. Your doctor will help determine the best treatment for you, and you may begin to feel better after. [substeps] Some people may not feel comfortable using an interpersonal therapy program.\nC. [substeps] Many forms of interpersonal therapy are used to help dependents or people with depression who have highs and lows. Participating in an interpersonal therapy session may help you better understand what to expect.\nD. [substeps] If you don't already have a therapist or psychologist, ask your doctor for a referral. You can also search online for therapists who treat your mental illness and use interpersonal therapy.","prompt":"[header] How to benefit from interpersonal therapy [title] Talk to your doctor. [step] The decision to use interpersonal therapy should be discussed with your doctor. If you have depression or a similar condition, you may decide to try interpersonal therapy prior to medication, or you may decide to combine medication with the therapy.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5697,"error":null,"output":"D"}
{"question_id":"hellaswag:00000558","item_index":192,"attempt":0,"prompt_hash":"9fe8c94dc063c4f38d92bc479839bf6533d04e443b1a22a466262d7cddd8ee92","question":"Person drives the red riding lawnmower across the yard. The dogs run across the yard. the person\n\nA. decides to turn off the red mower.\nB. pushes the mower as far as it will go and jumps over it.\nC. walks across the yard and opens the back door of the red riding lawnmower.\nD. puts a tack on a black, and then the dogs run by.","prompt":"Person drives the red riding lawnmower across the yard. The dogs run across the yard. the person","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6378,"error":null,"output":"C"}
{"question_id":"hellaswag:00000010","item_index":193,"attempt":0,"prompt_hash":"2355ecb6ac1d045201fdad03c4380015551e6953250c1e003dff1589ec7474c5","question":"Then he takes a piece of bark and rubs the powered stone pieces onto it. The stone particles stick to the wet piece of wood. he\n\nA. uses a small brush to polish one of the stone pieces.\nB. then uses a strip of sealing tape to seal the board and then smooths the bottom of the board.\nC. walks towards the camera.\nD. then takes the knife and sharpens it against the wood piece.","prompt":"Then he takes a piece of bark and rubs the powered stone pieces onto it. The stone particles stick to the wet piece of wood. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6378,"error":null,"output":"D"}
{"question_id":"hellaswag:00007704","item_index":194,"attempt":0,"prompt_hash":"2f0755a431b7aab5527074e76678e50471fa39f6ac8a8d2a3708c0d19eead506","question":"[header] How to use a baby bath tub [title] Decide if your baby needs a bath. [step] Babies, especially young babies, don't need to bath everyday. Before you decide to bathe your baby, take a few minutes to determine if they need a bath that day.\n\nA. If your baby needs a bath monthly, you can offer a bath at bedtime. If your baby needs a bath every 3-4 weeks, simply opt for a cup or two.\nB. Make sure your baby is well-fed and rests before you bathe them. Since babies can age very quickly, you don't want to have to wait until they get older to give them a bath.\nC. If they are not fussy, you may have to water them a couple of times, but that's okay. You can also create a bathing schedule to help you bathe your baby.\nD. Keep some of the following ideas in mind when deciding if you need to bathe your baby : [substeps] Babies under two to three weeks old should only have sponge baths. You can give a tub bath once the umbilical cord has fallen off and the stump has healed.","prompt":"[header] How to use a baby bath tub [title] Decide if your baby needs a bath. [step] Babies, especially young babies, don't need to bath everyday. Before you decide to bathe your baby, take a few minutes to determine if they need a bath that day.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6372,"error":null,"output":"D"}
{"question_id":"hellaswag:00004072","item_index":195,"attempt":0,"prompt_hash":"28ce741a6d2b39ef573071c3f2731deda01f28d135873d40a2b181719a7b0ccf","question":"[header] How to record a song at home easily [title] Obtain a home recording studio setup, you can use reflection filter products such as snaprecorder. [step] You will need this to record vocals. [title] Secondly make sure your computer has enough ram memory to run a daw (digital audio workstation).\n\nA. [step] This could be garageband, logic, cubase, protools or even audacity! [title] Organize what you want to record. [step] Guitars? Bass? Drums? Make sure you have the necessary equipment to record these.\nB. [step] All daws have a way to do this. [substeps] Have at least 2 gb of ram for audio, so your mp3s will not run as fast.\nC. [step] If you don't have enough ram memory then you need to purchase an extension, add noise meter, and up/down meter. [title] If you have more ram then keep a regular backup.\nD. [title] Acquire an ls4 and add max mc input and merc input. [title] Also make sure your microphone, playstation-player, and studio have enough microphone space in each place.","prompt":"[header] How to record a song at home easily [title] Obtain a home recording studio setup, you can use reflection filter products such as snaprecorder. [step] You will need this to record vocals. [title] Secondly make sure your computer has enough ram memory to run a daw (digital audio workstation).","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6371,"error":null,"output":"A"}
{"question_id":"hellaswag:00002439","item_index":196,"attempt":0,"prompt_hash":"a5c8253c58aecd1aae6721537310d5dfa624c7b452586488fdf89ea997430956","question":"A man and a woman run and exercise in a gym. the woman and the man\n\nA. run separately, the man brings do not move.\nB. star power on various jumps.\nC. talk in the gym and give advice if a person is cramped up.\nD. workout on working out in a gym.","prompt":"A man and a woman run and exercise in a gym. the woman and the man","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6377,"error":null,"output":"D"}
{"question_id":"hellaswag:00006389","item_index":197,"attempt":0,"prompt_hash":"2e1ecca4fad5f294ae1e863076b63e1f1ce5c2772dbc6f19503a128b92163392","question":"[header] How to follow customary behavior in church [title] If this is your first time at a particular church, or any church for that matter, recognize that you are a guest and new to this. [step] It is normal to not do everything right the first or the second time in a new church. [title] Many christian services are divided into two main components.\n\nA. [step] The first part of the service has to do with \" the word, \" in other words the bible. The second part of the service has to do with \" the table, \" and usually involves eucharist (communion).\nB. [step] Those, in particular, are the most common aspects of to follow. A typical vestry is a services that says, \" christmas day warms the christmas day heart.\nC. [step] First a prayer setting and second a spiritual setting. Avoid interfering with these arrangements, but be courteous in terms of courtesy.\nD. [step] The first is worship, and the second is worship. It is usually quite a formal event, and usually around easter, so expect to receive much praise for it.","prompt":"[header] How to follow customary behavior in church [title] If this is your first time at a particular church, or any church for that matter, recognize that you are a guest and new to this. [step] It is normal to not do everything right the first or the second time in a new church. [title] Many christian services are divided into two main components.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6372,"error":null,"output":"A"}
{"question_id":"hellaswag:00004917","item_index":198,"attempt":0,"prompt_hash":"3e63352204020a0e97c57bcf75e2a7825f06322fee814f4e0525d1271abd3e98","question":"[header] How to cut a shirt [title] Get your shirt. [title] Cut a slit into the shirt on either side of the collar using some sharp fabric scissors. [step] Cut the slit along the collar seam to serve as a visual guide for the rest of your cutting project.\n\nA. [title] Wash and lay the shirt (s) out. [step] Lay a sheet of newspaper in front of you.\nB. [substeps] You may notice the seam change while cutting, especially the collar. It will also be darker due to the stiffer, purer cut you will make.\nC. However, do not cut at either the neckline or sleeve seam. [substeps] Use thin nails, making sure to cut all the way through the shirt, not just the seam.\nD. [title] Use fabric scissors to cut the collar off of the shirt. [title] Grab the fabric at the shoulder seam and pull it.","prompt":"[header] How to cut a shirt [title] Get your shirt. [title] Cut a slit into the shirt on either side of the collar using some sharp fabric scissors. [step] Cut the slit along the collar seam to serve as a visual guide for the rest of your cutting project.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6371,"error":null,"output":"D"}
{"question_id":"hellaswag:00006609","item_index":199,"attempt":0,"prompt_hash":"5add5a2d9543c419250ff5eb25f418116db3aadd523bb47d6fc1978eba154591","question":"[header] How to test a starter solenoid [title] Open the hood of the vehicle. [step] The starter and solenoid are located on the engine of your vehicle. In order to gain access to it, pull on the hood release located near the door on the driver's side of the vehicle.\n\nA. [substeps] Depending on the engine type, the starter or engine will have a series of springs, clips, bolts, or a latch that allows you to slide out of the vehicle. If you're unsure, contact your vehicle's owners department to inquire.\nB. Loosen the release and locate the starter in the driver's side door of the vehicle. [substeps] In a manual transmission transmission, the starter and solenoid are both located on the rear windshield down side.\nC. [substeps] You will need to release the safety latch on the front of the vehicle in order to open the hood as well. If you are unable to locate the safety release, refer to your vehicle's owner's manual for directions.\nD. If you can't access the hood release of the starter, you'll need to get it out of the way to prevent it from breaking through the driver door. [substeps] You can open a vehicle's hood or retrieve the starter element by pulling down on the top latch on the vehicle's dashboard.","prompt":"[header] How to test a starter solenoid [title] Open the hood of the vehicle. [step] The starter and solenoid are located on the engine of your vehicle. In order to gain access to it, pull on the hood release located near the door on the driver's side of the vehicle.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6375,"error":null,"output":"C"}
{"question_id":"hellaswag:00004116","item_index":200,"attempt":0,"prompt_hash":"c6a0d72c246abaa1494f251952c7ec66dc91602005874eaeffdb8f8eb4aa68e8","question":"[header] How to clean pergo floor [title] Clean the floor regularly with a damp dust mop. [step] Pergo floors will attract dust, so you'll want to make sure you wipe it clean regularly with a dust mop to remove the most obvious particles. A regular wipe with a dry mop or static cloth should get most of the dust.\n\nA. [substeps] Place it flat on the floor with the dust mop facing up from below. If you don't have access to a dust mop, a broom can be used instead.\nB. [substeps] Before wiping down the floor, wet your dust mop slightly. A bucket of water nearby should be enough to keep the mop moist.\nC. [substeps] Don't just wipe the floor down. Do too much and you risk damaging the floor-you can sweep up any dust now.\nD. [substeps] Pergo flooring will also come in handy if you run out of spare space in your home or if you're moving furniture in your home that needs cleaning. [title] Clear a lot of dust from the floor with a soft cloth.","prompt":"[header] How to clean pergo floor [title] Clean the floor regularly with a damp dust mop. [step] Pergo floors will attract dust, so you'll want to make sure you wipe it clean regularly with a dust mop to remove the most obvious particles. A regular wipe with a dry mop or static cloth should get most of the dust.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6370,"error":null,"output":"B"}
{"question_id":"hellaswag:00006536","item_index":201,"attempt":0,"prompt_hash":"6187725c4b694967f870cb2629b17183e801dc93591f083ce36d3c4f5ee70e73","question":"[header] How to grow vegetables [title] Decide if you want to plant vegetables in the ground, in raised beds, or in containers. [step] Each of these options has its own advantages and disadvantages, so consider your situation before you decide to determine which method will work best for you. [substeps] Planting in the ground is great if you have good soil and don't mind getting down on your hands and knees.\n\nA. It is great if you prefer to do it in open space as long as it gets enough sun. You can grow vegetables underground or from the floor of the house.\nB. It is often the best option if you want to grow large amounts of vegetables. Raised beds work well if you don't have very good soil and/or you have a bad back.\nC. Planting in a raised bed is a great way to plant vegetables without a lot of soil. It works by transferring oil or clay into the soil and encourages oxygen to build up, and helps the plants grow well.\nD. Planting in the ground makes your soil easier for vegetables to grow in. It can also prevent the budding of vegetables by exposing them to sunlight.","prompt":"[header] How to grow vegetables [title] Decide if you want to plant vegetables in the ground, in raised beds, or in containers. [step] Each of these options has its own advantages and disadvantages, so consider your situation before you decide to determine which method will work best for you. [substeps] Planting in the ground is great if you have good soil and don't mind getting down on your hands and knees.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6370,"error":null,"output":"B"}
{"question_id":"hellaswag:00009573","item_index":202,"attempt":0,"prompt_hash":"653dcc8a665d42e32d4caead16962c42e5db79ce1bf49ee0734b888efe3ec42a","question":"[header] How to succeed in college with young children [title] Set a daily routine. [step] Having a set routine helps create some form of normalcy, a sense of security, and it establishes everyday expectations. Planning set times for dinner, story time, bath time, bed time, etc.\n\nA. This gives your children some form of routine they can expect to have at bedtime every night. [substeps] If you can't make your own schedule, have the children make one.\nB. Ensure your child (and/or caretaker) these things. [title] Try don't overeat.\nC. Not only does this help your child adjust to your new lifestyle but it helps you succeed in college. [substeps] Consider writing out a family schedule for all to see.\nD. The routine can help regulate your mood and can help you think through everything, even when you're feeling tired or frustrated. [substeps] This is especially important with housebreaking, as it's when a child reaches his earliest developmental stage that most independence is gaining.","prompt":"[header] How to succeed in college with young children [title] Set a daily routine. [step] Having a set routine helps create some form of normalcy, a sense of security, and it establishes everyday expectations. Planning set times for dinner, story time, bath time, bed time, etc.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6370,"error":null,"output":"C"}
{"question_id":"hellaswag:00003685","item_index":203,"attempt":0,"prompt_hash":"5b7690f82463efe06f0e260af16c0ddb50a3635ecebf7142be68a51bb69f0297","question":"[header] How to wear a sport coat [title] Learn the difference between a sport coat and other coats. [step] Though they're commonly conflated, a sport coat is not a blazer or a suit jacket. Sport coats need not match the fabric of the pants, as you'd find with a suit jacket.\n\nA. The sport coat should also wear dark wash jeans and a tie. [substeps] Sports coats especially stick out in the summer and stay in place at night during the day.\nB. Instead, they have a specific pattern or color used in place. Once you find a sport coat, stay in mind that it needs to be closely tailored.\nC. In fact, many sport coats are used to cover up \" casual \" or something too formal, like a suit or slacks. Sport coats typically go below the knee or above.\nD. The biggest difference between a sport coat and a blazer is that sport coats are patterned, while blazers are made of solid fabric with a contrasting color of button. [substeps] Stylistically, sport coats are sometimes a slightly looser fit than other kinds of dress coats, allowing for outdoorsy use and \" sporting.","prompt":"[header] How to wear a sport coat [title] Learn the difference between a sport coat and other coats. [step] Though they're commonly conflated, a sport coat is not a blazer or a suit jacket. Sport coats need not match the fabric of the pants, as you'd find with a suit jacket.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6370,"error":null,"output":"D"}
{"question_id":"hellaswag:00009797","item_index":204,"attempt":0,"prompt_hash":"bd2054e911aea6d7944ff3c9b980645156f2784cac3d6a5e144c3d536ed2feed","question":"[header] How to address abuse in polyamory [title] Understand the various manifestations of abuse. [step] Become informed about common (and not so common) ways that abuse can manifest in polyamory. This is one way you can protect yourself and others against it.\n\nA. [substeps] Polyamory is especially important for individuals living in abusive relationships. If you are an individual, have a strong need to protect yourself from abuse.\nB. Make detailed notes on all of your sessions, so that you can replace any reference to abusive abuses with the appropriate language and case law. [substeps] Find a directory at http: // www.blackhouse.\nC. Read as much as you can, and most importantly, trust your gut. Below are some examples of poly-specific abuse patterns.\nD. [substeps] Men can develop self-starvation at a young age due to the feeling of powerlessness that there is never enough food to fulfill their needs. Women get abused at an early age due to the feeling of isolation and helplessness around the age of five when they are capable of full functioning.","prompt":"[header] How to address abuse in polyamory [title] Understand the various manifestations of abuse. [step] Become informed about common (and not so common) ways that abuse can manifest in polyamory. This is one way you can protect yourself and others against it.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6372,"error":null,"output":"C"}
{"question_id":"hellaswag:00009451","item_index":205,"attempt":0,"prompt_hash":"0d8a767d948a02f7a654d1ca2d0f254b9648a471581dae0570ba83525864af4f","question":"[header] How to change wavy frizzy hair to straight hair [title] Rough dry your hair. [step] If you already blow dry your hair, the way you position your blow dryer may be contributing to your frizzy hair. When drying your hair, shake the dryer over your head, back and forth, until your hair is 80% dry.\n\nA. [substeps] Keep your blow dryer's nozzle face down, otherwise your hair will frizz. When hair is 80% finished, apply a smoothing or anti-frizz serum, before completing your blow out.\nB. When sitting down, gather your hair in the palm of your hand and scrunch it for three to five minutes. [title] Use a food processor or blender to blend your hair.\nC. This article will also refer to hair softening methods. [substeps] Your hair may still remain shinier after it is dry.\nD. At this point, your hair should feel almost like wet noodle hair. [title] Apply the mousse to damp your hair while it is still damp.","prompt":"[header] How to change wavy frizzy hair to straight hair [title] Rough dry your hair. [step] If you already blow dry your hair, the way you position your blow dryer may be contributing to your frizzy hair. When drying your hair, shake the dryer over your head, back and forth, until your hair is 80% dry.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6372,"error":null,"output":"B"}
{"question_id":"hellaswag:00009636","item_index":206,"attempt":0,"prompt_hash":"add27395431efade598ab5ad343c6ef46df5f6c6aded5b2c165575af6eff514d","question":"[header] How to bake kabocha squash [title] Cut the stem off the top of the squash. [step] Place the tip of a sharp kitchen knife 1 inch (2.5 cm) from the stem of the squash. Penetrate the skin of the squash and cut around the stem.\n\nA. Lift the peel off the top of the squash by turning the knife counterclockwise and keeping pressure on the peel. Tear off the top of the squash by turning it sideways with a sharp knife.\nB. Pull the tender sections from the stem and discard the central core. [substeps] Sometimes you'll need to water the squash to dry it out.\nC. Then, pull up on the stem to remove it. [substeps] You need a sharp knife to cut through the thick skin of the squash.\nD. [substeps] If possible, have someone else cut the squash up to ensure that it is safe to be cut into pieces. [title] Boil the squash to turn it into bite-sized pieces.","prompt":"[header] How to bake kabocha squash [title] Cut the stem off the top of the squash. [step] Place the tip of a sharp kitchen knife 1 inch (2.5 cm) from the stem of the squash. Penetrate the skin of the squash and cut around the stem.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6373,"error":null,"output":"C"}
{"question_id":"hellaswag:00002242","item_index":207,"attempt":0,"prompt_hash":"ca0cf0bd4d0c572a23b47deb17cb1fe3f665891e7d8e410989379ef57f7d567a","question":"Two young girls are in a gym and they do a side ways hurtles over a bar in the middle of the gym. the video\n\nA. showed a group of girls climbing a balance beam and they did all the simple exercises and then they start doing more.\nB. rolls back high and the three girls still go up and over the bar and there is still no sign of the girl who did the top spin.\nC. is then replayed in slow motion and once again in a faster speed.\nD. is black and white including some different titles.","prompt":"Two young girls are in a gym and they do a side ways hurtles over a bar in the middle of the gym. the video","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6373,"error":null,"output":"C"}
{"question_id":"hellaswag:00000282","item_index":208,"attempt":0,"prompt_hash":"20e409a78702bb0f09076804344134b48edacde035bcca4d4257f2b4353a076b","question":"We see a man standing in a room. The man starts performing tai chi. The man turns and faces left. the man\n\nA. turns to face right.\nB. starts dancing for the camera.\nC. demonstrates tai chi in a seated position.\nD. does a squat and drinks, then turns and walks from the room.","prompt":"We see a man standing in a room. The man starts performing tai chi. The man turns and faces left. the man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6373,"error":null,"output":"A"}
{"question_id":"hellaswag:00004761","item_index":209,"attempt":0,"prompt_hash":"74505f85236fb8f07f80ced461a1f69975009fafbcc1c5ee8c05df6617794351","question":"[header] How to use a lice comb on dry hair [title] Buy a lice detection comb. [step] You'll need to get a special fine-toothed comb to detect lice. You can buy the comb from your local pharmacy, medical supply store, grocery store or large retailer.\n\nA. [substeps] Read the label to make sure that you have a lice comb. This type of comb will have tooth spacing of 0.2-0.3 mm in order to capture the smallest lice.\nB. They come in many different sizes and types to make you a better choice for use. [substeps] Comb combs are tiny tweezers used to lift lice.\nC. [substeps] To find a good-toothed comb, look for a comb with a curved or curve. The curved or curve may make it easier to identify lice.\nD. You could also buy a special lice-removal brush for wet hair. [substeps] Lice combs work best in wet hair.","prompt":"[header] How to use a lice comb on dry hair [title] Buy a lice detection comb. [step] You'll need to get a special fine-toothed comb to detect lice. You can buy the comb from your local pharmacy, medical supply store, grocery store or large retailer.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6369,"error":null,"output":"A"}
{"question_id":"hellaswag:00004189","item_index":210,"attempt":0,"prompt_hash":"9e370fae8720e0b0e32debb61fc754c7f1d6615d7a57a0e44d183b852b26852e","question":"[header] How to make your toddler laugh [title] Play peekaboo. [step] Babies and toddlers typically enjoy and find humor in being surprised. When you are playing with your child, take your hands and place them over your eyes and then remove your hands and say' boo! ' you can even encourage your child to play too by having them place their hands over their own eyes instead.\n\nA. [substeps] You may find that your toddler prefers a peeking child version of peekaboo. You can then play peekaboo while your child is sleeping or snoozing.\nB. [substeps] Pretend to laugh while your hands are over your eyes. Your toddler will likely be amazed at how silly you look when they learn that their hands are not touching your face.\nC. [substeps] This works particularly well when you are physically very close to your child or they are seated in your lap. Try not to hide your face for too long, or else you might stress out the child.\nD. This also works best if your toddler feels like he or she has been swiftly told to play peekaboo. [title] Find some treats for your toddler.","prompt":"[header] How to make your toddler laugh [title] Play peekaboo. [step] Babies and toddlers typically enjoy and find humor in being surprised. When you are playing with your child, take your hands and place them over your eyes and then remove your hands and say' boo! ' you can even encourage your child to play too by having them place their hands over their own eyes instead.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6372,"error":null,"output":"C"}
{"question_id":"hellaswag:00008574","item_index":211,"attempt":0,"prompt_hash":"0e7d4e8852f3019cdc46f4b076c9af94ef137dfe495bcef00461b083ad4a2a41","question":"[header] How to restore a boat [title] Make a plan. [step] Plan your work by breaking everything down into sections, like chapters in a book. Chapter one is planning.\n\nA. This is where your business development can start. Analyze the navigation board, take all the necessary materials and then create your grand vision.\nB. This step is not yet complete, so you need to prioritize moving toward part 1 of the project. This will help you tackle all your major projects faster, so that you can review your own work by taking each section and remembering to include the steps in that section over time.\nC. The next phase might be stripping the boat in preparation for the work. [title] Prepare a work site, gather tools and supplies, carve out a dedicated time on a regular basis with your schedule, consult experts in the field, and also outsource certain components as needed to certain experts.\nD. Plan your time on your own to avoid confusion and frustration with each chapter. Be sure that you aren't scrambling for ideas as you go forward so that you don't get overwhelmed by too many ideas.","prompt":"[header] How to restore a boat [title] Make a plan. [step] Plan your work by breaking everything down into sections, like chapters in a book. Chapter one is planning.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6374,"error":null,"output":"C"}
{"question_id":"hellaswag:00008351","item_index":212,"attempt":0,"prompt_hash":"92619c4265b5f07ea6bd1b67de644959448f278773a1d21eebd06424b1e1c4ec","question":"[header] How to sew a sock dog [title] Select a suitable sock. [step] You'll need a good quality, long sock. It needs to be long, to accommodate the dog's body.\n\nA. The longer, the better as it will provide all the pieces you need. [substeps] If the sock proves to be too short, you will need to use scrap materials for the ears, tail and legs.\nB. For the sock dog, choose one big, one medium size sock, and 1 mini sock, for a pair of socks made to fit all sizes. As you become more frustrated, start looking around.\nC. [substeps] While a true sock dog might be able to dance and do most odd cartwheels, there are some knots that can make your dog hop up in the air as if it was the witch running around town instead. The o and o solution mixture is easily available on the internet.\nD. [title] Prepare the sock pup's lining. [step] Lay the sock pup down on a clean surface and cut off the very edge of the paper, so that you have a piece that matches the sock.","prompt":"[header] How to sew a sock dog [title] Select a suitable sock. [step] You'll need a good quality, long sock. It needs to be long, to accommodate the dog's body.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6371,"error":null,"output":"A"}
{"question_id":"hellaswag:00006117","item_index":213,"attempt":0,"prompt_hash":"23dea1acfcffaf829b71d783f8e416552dae6544cc44a83be0b2cd5eae39c6d1","question":"[header] How to encourage your senior dog to play [title] Play games with treats. [step] One way to encourage your senior dog to play is by incorporating treats into the game. Most dogs are motivated by food and will happily play food related games.\n\nA. Some of the easiest and most stimulating games that you can play with older dogs include foraging for food. These games will not only encourage your older dog to get up and moving, but they will also stimulate their brains allowing them to problem solve.\nB. Choose a game and expect that your senior dog will soon start to play. [substeps] If your dog suddenly starts to play fetch, play other games with it like fetch or charades, where you will hold out treats for your dog as it plays, but then practice tug or tug.\nC. This can also be a fun way to socialize your dog while providing some outdoor entertainment. [substeps] If the senior dog or the older dog makes a mess, try out a few homemade games to encourage them to play.\nD. When you put together your dog's favorite toys, you should be sure to emphasize one at a time. For example, if your dog only has one toy, you may want to reward it with a treat every time the toy is a treat you place inside your dog's mouth or you feed it new kinds of treats.","prompt":"[header] How to encourage your senior dog to play [title] Play games with treats. [step] One way to encourage your senior dog to play is by incorporating treats into the game. Most dogs are motivated by food and will happily play food related games.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6373,"error":null,"output":"A"}
{"question_id":"hellaswag:00006222","item_index":214,"attempt":0,"prompt_hash":"af4f784d630974da0a19f767e44998e051a40beee12b64d61200ddd1ac376c51","question":"[header] How to remove set in stains [title] Purchase an oxygen bleach product. [step] There are many to choose from, but oxiclean is probably the most easily recognized. The specific product you buy is your preference, but you should look for a powdered formula that contains sodium percarbonate and sodium carbonate.\n\nA. You will need at least four gallons of oxygen bleach per gallon to use your child's bathroom, although higher volumes of oxygen will generally be used if you have more children. Buy a suitable formula from a hardware store or online.\nB. At minimum, you should buy the product with 1% sodium percarbonate. A powdered formula does the same thing, but you don't have to worry about the chemical reaction going on too much.\nC. Other fillers will reduce the efficacy of the product. [substeps] Some other recommended brands are biokleen's oxygen bleach plus, oxo-brite (which is supposed to be earth-friendly), and vaska's oxygen brightener.\nD. This will help keep the stain away from you and keep it from infecting the entire garment. [title] Start with a clean cloth and a bit of bleach.","prompt":"[header] How to remove set in stains [title] Purchase an oxygen bleach product. [step] There are many to choose from, but oxiclean is probably the most easily recognized. The specific product you buy is your preference, but you should look for a powdered formula that contains sodium percarbonate and sodium carbonate.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7029,"error":null,"output":"B"}
{"question_id":"hellaswag:00005293","item_index":215,"attempt":0,"prompt_hash":"926e8c65526ab4a8b8405ad052400fe41ea7dcebfb5ef188695fcd2e4ac81d39","question":"[header] How to burn letters into wood [title] Pick out wood. [step] Any wood surface can be. However, some are better than others.\n\nA. Do some research about the different types of wood you have so you can determine which material will work best for your project. [substeps] Make sure the wind in your area and temperatures are acceptable before roasting your letters.\nB. Light-colored and soft wood, such as bass, works especially well. This is because the wood burning stands out on the light color and you don't have to push very hard to make your marks.\nC. Any surface that is more akin to wood (like cobblestone, concrete, or soda is for instance) works. [substeps] Look at the grain of the wood, and if you can't find that grain, use a compass or a straightedge to find a grain grain.\nD. The first step to burning wood is weighing it down in the right place. You want to make sure you have something either loose on the surface, or that doesn't suck up very much of the wood, especially a pen or pencil.","prompt":"[header] How to burn letters into wood [title] Pick out wood. [step] Any wood surface can be. However, some are better than others.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7030,"error":null,"output":"B"}
{"question_id":"hellaswag:00003964","item_index":216,"attempt":0,"prompt_hash":"dcd95f6ff16b0c5f33e8ad16c56fcfed23e24e963e9ce83cc2a53f695b44f193","question":"[header] How to wear a bodysuit [title] Pick a bodysuit that flatters your favorite feature. [step] With so many options and styles, the bodysuit can truly be universally flattering. To find a bodysuit that looks great on you, consider what part of your body you want to highlight.\n\nA. This could be the underarms, legs, or any places else that stick out. For example, maybe you have a nice cleft lip and would like to highlight those.\nB. Figure out what part of your body you want to highlight, and then cut down what accentuates that part. For example, if you want the tops to emphasize your breasts to the fullest, the bikini bottoms can be worn around that area too.\nC. [substeps] For example, if you're proud of your toned arms, opt for a sleeveless or halter-neck bodysuit. [title] Start with a t-shirt style bodysuit if you're warming up to the trend.\nD. [substeps] Decide how many different parts of you that you want your bodysuit to highlight. For example, if you want a sporty look, your bodysuit should also highlight one part of your body instead of one particularly noticeable part.","prompt":"[header] How to wear a bodysuit [title] Pick a bodysuit that flatters your favorite feature. [step] With so many options and styles, the bodysuit can truly be universally flattering. To find a bodysuit that looks great on you, consider what part of your body you want to highlight.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7028,"error":null,"output":"C"}
{"question_id":"hellaswag:00006792","item_index":217,"attempt":0,"prompt_hash":"f4bdd504d79f1e16259844342c8d079d484f49ed9a5c9db30dcb769f2c56e4b2","question":"[header] How to teach your kid to hold a pencil [title] Get the right pencil for your child. [step] Using short pencils, like golf pencils or normal pencils broken in half, can help children find the right grip on their own. Short pencils have less room for unnecessary fingers, so your child has little choice but to use the correct three-finger grip.\n\nA. [substeps] You can purchase short pencils at most craft and office supply stores. [title] Help your child out with a pencil grip.\nB. [substeps] Good pencils to start with include the \" p \" and \" g \" pencils. Start in the bottom center pencil holder, and watch your child closely as he tries to hold the pencil in place.\nC. [substeps] For more information on how to handle a pencil, visit. [title] Differentiate between pencils that are powered by the hands and pencils that are used with both hands.\nD. [substeps] A finger with a longer pencil point allows children to hold pencils with much more control. A finger with lower points is available for trying to control just about any finger position.","prompt":"[header] How to teach your kid to hold a pencil [title] Get the right pencil for your child. [step] Using short pencils, like golf pencils or normal pencils broken in half, can help children find the right grip on their own. Short pencils have less room for unnecessary fingers, so your child has little choice but to use the correct three-finger grip.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7029,"error":null,"output":"A"}
{"question_id":"hellaswag:00004363","item_index":218,"attempt":0,"prompt_hash":"1fb337968671bf53ea6bc3a2684a6a2aff4715bf1154a601271f760e605c1bac","question":"[header] How to deal with relatives who undermine your parenting [title] Communicate your house rules. [step] There's a chance that others are undermining your parenting because your intended rules aren't clear. You can't expect in-laws or others to respect your parenting guidelines when they don't fully understand your goals.\n\nA. Tell your parents things you disagree with and set ground rules that address those issues. [substeps] You might not be as willing to compromise if you live in a household where the family is less organized and is more confrontational.\nB. Take the time to communicate your rules to those who spend time with your children. This may prevent any undermining down the road.\nC. [substeps] Find some common ground where arguments and disputes will be settled on some basis. Decide on a firm, consistent policy like no reference: no dates are set, and no guests are out until more than three months after the parents have the argument.\nD. However, understanding a set of rules is very important if you want to pursue a productive marriage and family life. [substeps] Briefly identify the standards for a household environment, including rules about time, respect for space, and make your expectations clear so your partner doesn't resent your parenting decisions.","prompt":"[header] How to deal with relatives who undermine your parenting [title] Communicate your house rules. [step] There's a chance that others are undermining your parenting because your intended rules aren't clear. You can't expect in-laws or others to respect your parenting guidelines when they don't fully understand your goals.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7028,"error":null,"output":"B"}
{"question_id":"hellaswag:00007710","item_index":219,"attempt":0,"prompt_hash":"28c00797b5285ade020305f977827ce08c57527210f2540646c6717cb698fba5","question":"[header] How to redeem your repossessed car [title] Find your deadline. [step] After the lender repossesses the car, it should send you some notices in the mail. One notice should inform you of how you can redeem your car.\n\nA. [title] Determine how much money you have saved. [step] If you want to redeem your repossessed car, you might want to shop around for repossessed cars so that you can save as much as possible.\nB. The notice should also tell you the deadline for redeeming the car and the redemption amount. If you don't receive this notice within five days of repossession, then contact the lender.\nC. Typically, the notice should mention whether the lender repossesses the car as a new or used one. The notice should also say what the dates are for purchasing a repossessed vehicle in the future.\nD. These notices must include dates you will redeem and instructions for getting the car's repossessed vehicle. [substeps] See method 1 for detailed instructions on how to redeem a repossessed car in the future.","prompt":"[header] How to redeem your repossessed car [title] Find your deadline. [step] After the lender repossesses the car, it should send you some notices in the mail. One notice should inform you of how you can redeem your car.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6964,"error":null,"output":"B"}
{"question_id":"hellaswag:00004208","item_index":220,"attempt":0,"prompt_hash":"1e8c21fce83b5ef19668fc216e59b19cc041dc4035f875d9ec7779bfe2859705","question":"[header] How to be a new york mets fan [title] Start watching mets games. [step] Pay attention to the whole game, don't drift in and out. Decide which player you like the most by watching the plays he makes on the field and his interviews after the game (their personality is important too! ).\n\nA. [title] Perform cpr at the end of any matches. [step] Double check your id when the game is over when asked to perform the cpr.\nB. It's vital that you can play in front of friends and family, but in general you like to watch the games anyway, so it's easy to become a fan. [substeps] Get familiar with the phrase \" mets \" and how it says \" mets rivalry \".\nC. [title] Get into your favorite sports league and watch the mets games, but don't watch them with someone else. [step] If you see the mets team during the movie, vomit it.\nD. [title] Learn about the mets history (they won the world series in 1969 and 1986, were created in 1962, etc) and research some team trivia. [step] This can help you hold your own in conversations with even lifelong die-hards.","prompt":"[header] How to be a new york mets fan [title] Start watching mets games. [step] Pay attention to the whole game, don't drift in and out. Decide which player you like the most by watching the plays he makes on the field and his interviews after the game (their personality is important too! ).","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6964,"error":null,"output":"D"}
{"question_id":"hellaswag:00007378","item_index":221,"attempt":0,"prompt_hash":"7deae07e21f870c2edec611b960790b42a989c80992cd86e8ac67dd40de33f80","question":"[header] How to show your parents you love them [title] Ask them about their life experiences. [step] It's sometimes difficult for a child to remember that their parents have lives, too. What's more, it can be hard to believe that their lives didn't always revolve around you and your siblings.\n\nA. To show that you love them, you might ask them questions about their daily lives and share your own experiences. [substeps] In the case of older siblings, you might ask, \" so, what's your biggest life experience? How did your parents handle it? \" this will show that you want to know.\nB. Sitting down with your parents and asking them about their lives shows you care and take an interest in them. You may even learn something about them you never knew before.\nC. By asking about your parents' lives, you'll be immediately proved as kind, respectful, and caring. [substeps] Ask them honestly about what they do and why they're doing it.\nD. Use your own experience to show that you do still have a caring, loving family. [substeps] If your parents are struggling to remember to live, ask them.","prompt":"[header] How to show your parents you love them [title] Ask them about their life experiences. [step] It's sometimes difficult for a child to remember that their parents have lives, too. What's more, it can be hard to believe that their lives didn't always revolve around you and your siblings.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6964,"error":null,"output":"B"}
{"question_id":"hellaswag:00005524","item_index":222,"attempt":0,"prompt_hash":"40301a7304749e024711b3a717834fd192292787eb30d333a39c8243bb4a1264","question":"[header] How to add hyperlinks to email messages in hotmail [title] Visit the hotmail website in your web browser. [title] Type the required info into the appropriate boxes to sign-in (if not already completed and saved for further login). [title] Click the \" go to inbox \" link listed directly under \" hotmail highlights \".\n\nA. [title] Click the \" tools \" button to go to the \" productivity tools \" tab. [step] Click on \" deleting emails \".\nB. [step] You will now receive emails from multiple people using the same service. [title] Look at the cursor on the upper-right corner of the screen.\nC. [title] Click the \" view or : [step] \" button from the left menu bar , [title] Select an email subject from the main menu, then click the \" open text now \" button. [title] Click the \" log in \" button that is beneath the mail icon on the pop-up.\nD. [substeps] By preference, try to learn and use the (http: // www. ) portion of the address link formatting, when typing your link.","prompt":"[header] How to add hyperlinks to email messages in hotmail [title] Visit the hotmail website in your web browser. [title] Type the required info into the appropriate boxes to sign-in (if not already completed and saved for further login). [title] Click the \" go to inbox \" link listed directly under \" hotmail highlights \".","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6965,"error":null,"output":"B"}
{"question_id":"hellaswag:00005902","item_index":223,"attempt":0,"prompt_hash":"db662a06e1698cc13b0ad77d03ccada7ae0ed1594f2a3e1befadf14f536e3d37","question":"[header] How to advertise on ebay daily deals [title] Maintain a high positive feedback rating. [step] 98 percent of your feedback ratings from buyers must be positive for 12 consecutive months, as this factor demonstrates buyer satisfaction. [substeps] Ship your products on time and provide honest, accurate information in your listings to keep your buyers satisfied with their purchases.\n\nA. Make sure you are using your direct online analytics. Take time to review your sales and counter-offers to understand the best deal for your customers.\nB. [title] Follow ebay's rules and policies. [step] This practice will demonstrate that you support ebay's business ethics and procedures.\nC. [title] Operate with a reliable and reputable brand name. [step] This will prevent your brand from being merged into a wholesale product like ebay.\nD. [title] Use online advertising methods when creating a ranking chart that can accommodate large number of sellers. [substeps] In the example above, a seller that has 12 successful deals is probably more successful on ebay's ranking chart than being on ebay.","prompt":"[header] How to advertise on ebay daily deals [title] Maintain a high positive feedback rating. [step] 98 percent of your feedback ratings from buyers must be positive for 12 consecutive months, as this factor demonstrates buyer satisfaction. [substeps] Ship your products on time and provide honest, accurate information in your listings to keep your buyers satisfied with their purchases.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6965,"error":null,"output":"B"}
{"question_id":"hellaswag:00006017","item_index":224,"attempt":0,"prompt_hash":"3dd0fc38944d9535557e414b1af65e3ad5b8cc39c74c57e447ce9bb442ba9911","question":"[header] How to fix pelvic tilt [title] Foam roll your pelvic area each day. [step] Place a foam roller on an exercise mat. Then, lay down on your side on the mat with the roller touching your hip area.\n\nA. Then, roll your hips towards the roller. Over time, you will strengthen your pelvic area by rolling in one direction.\nB. With the foam roller, roll your pelvic area in one direction for 30-45 seconds to increase blood flow to your pelvic area. [substeps] If you don't have a foam roller, you can buy or build your own.\nC. Support your head with a bent arm and bend your knees for stability, too. Use your feet to gently push your body up and down the roller around the hip area.\nD. Slowly roll the roller slowly on the area for four to five minutes, preferably with 1-2 hours of sitting on it. Then, stop rolling as soon as it begins to lean towards your hip.","prompt":"[header] How to fix pelvic tilt [title] Foam roll your pelvic area each day. [step] Place a foam roller on an exercise mat. Then, lay down on your side on the mat with the roller touching your hip area.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7136,"error":null,"output":"C"}
{"question_id":"hellaswag:00007275","item_index":225,"attempt":0,"prompt_hash":"7f3e735d9b0f71d181bf0dad6bf097119019757cfa8cd8ca00b8654355720301","question":"[header] How to make fabric covered beads [title] Select the beads. [step] Most beads will work when covered, as long as they're not rough or precious. Choose the beads in sizes that are easy to cover; in other words, not too small and it's probably a good idea to avoid beads with odd shapes, as they may be harder to define after being wrapped in fabric.\n\nA. [substeps] You may want to vary bead sizes, depending on the design for your handmade jewelry. [title] Select the fabric to cover the beads according to your needs.\nB. [substeps] Select beads that are made of fine-grain or super fine grain. This is the most convenient method to ensure good adhesion.\nC. [title] Use hot glue and pipe cleaners to secure the beads to the fabric. [step] For vintage beads, you may need to use a little bit of foam, craft glue or even elmer's glue.\nD. [title] Brush a suitable, white or ivory bristled brush across the inner arms of the beads. [step] This will ensure an even look on the beads.","prompt":"[header] How to make fabric covered beads [title] Select the beads. [step] Most beads will work when covered, as long as they're not rough or precious. Choose the beads in sizes that are easy to cover; in other words, not too small and it's probably a good idea to avoid beads with odd shapes, as they may be harder to define after being wrapped in fabric.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7136,"error":null,"output":"A"}
{"question_id":"hellaswag:00009725","item_index":226,"attempt":0,"prompt_hash":"478c819336ed30cd2a17011ffb6f4791181a498e4589dc2c3a92e727141980fa","question":"[header] How to evaluate the protein silk fusion hair attachment method [title] Ask yourself about the condition of your natural hair. [step] Be honest with yourself and you may be able to rule out protein silk infusions as a good match for your situation. [substeps] Protein silk fusion is ideal for those who have healthy hair.\n\nA. As we all know, we are usually the result of deep dermatitis. But if your hair follicles are dark or stained with a deep nutrient for protein silk production then it is a sign that protein silk fusion is not a good match for you or your hair as an individual.\nB. [title] Be aware that the protein silk only performs functions on the strands of hair. [step] It can't replicate the proportions of the protein silk which may be a counter in the statement.\nC. However, if you have severely damaged hair that needs extra care, such as dry hair or excessively waxed hair, this method should be avoided. Protein silk infusions occur only after your hair is completely dry.\nD. If you have had recent chemical treatments or have any visible damage, protein silk infusions are probably not right for you. You should carefully examine other methods before returning to further consider this method.","prompt":"[header] How to evaluate the protein silk fusion hair attachment method [title] Ask yourself about the condition of your natural hair. [step] Be honest with yourself and you may be able to rule out protein silk infusions as a good match for your situation. [substeps] Protein silk fusion is ideal for those who have healthy hair.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7136,"error":null,"output":"D"}
{"question_id":"hellaswag:00008247","item_index":227,"attempt":0,"prompt_hash":"605508922367def6506c8f51d53cc3db0a6d95a4890665668fc771e14c2a3d5e","question":"[header] How to balance breast size during breastfeeding [title] Nurse on the smaller breast first. [step] Your baby's suckling is what triggers your body to produce milk. Also, babies tend to suckle harder when they first start feeding; therefore, if your baby suckles on the smaller breast first when you feed them, it can help trigger milk flow in that breast, helping to balance out the size of your breasts.\n\nA. [substeps] Pour small amounts of milk from your baby's breast into the cup on your left hand. Slowly pour milk over your hands until your hands are full.\nB. [substeps] This solution only works if one breast is producing normally while the other is under producing. If one is overproducing, you may need to drain the milk off to prevent engorgement.\nC. [title] Determine how much milk you want to give. [step] Since your milk is produced by fermentation and feeding, your milk will function differently.\nD. [substeps] Trim a forearm area while feeding your baby first if you have one. You can do this with a pad, nursing stick, or a diaper on.","prompt":"[header] How to balance breast size during breastfeeding [title] Nurse on the smaller breast first. [step] Your baby's suckling is what triggers your body to produce milk. Also, babies tend to suckle harder when they first start feeding; therefore, if your baby suckles on the smaller breast first when you feed them, it can help trigger milk flow in that breast, helping to balance out the size of your breasts.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7137,"error":null,"output":"B"}
{"question_id":"hellaswag:00002784","item_index":228,"attempt":0,"prompt_hash":"f8b8890df0f44c963d2018e0a900fe754440f372076331e277152f6fe73c628c","question":"A man is shooting a basketball into a hoop. another man\n\nA. is holding a racket and is shooting hoops into the hoop behind him.\nB. starts shooting a basketball into the hoop.\nC. in black is shown practicing with a tennis racket.\nD. is in a room shooting basketballs.","prompt":"A man is shooting a basketball into a hoop. another man","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7139,"error":null,"output":"B"}
{"question_id":"hellaswag:00001812","item_index":229,"attempt":0,"prompt_hash":"84492a325b5e264091356a0c329eed6ab56e314fc5695968ced2bc03661cd154","question":"A man touches a knife on a table. He picks up a sharpener and sharpens the blade of the knife on the sharpener. he\n\nA. demonstrates how much sharper the knife is by cutting paper in between attempts to sharpen it.\nB. picks up a dabbler and rubs the sharpener in.\nC. sharpens the knife on the end of the sharpener.\nD. shows off the tools he uses.","prompt":"A man touches a knife on a table. He picks up a sharpener and sharpens the blade of the knife on the sharpener. he","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7139,"error":null,"output":"C"}
{"question_id":"hellaswag:00007769","item_index":230,"attempt":0,"prompt_hash":"e94739ac965472b752b2d3852688d7e39a6b04cac09839f9ec1722575ed61b54","question":"[header] How to introduce wheat to your baby [title] Introduce solids at 4 months. [step] If your child is transitioning from liquid foods to solids, then you can start feeding them wheat at this time as well. Most children start this transition between 4-7 months.\n\nA. Adding wheat into your baby's diet at this point helps to give them additional calories and nutrients, such as folate. [substeps] Giving your baby potentially allergenic foods, such as wheat or eggs, early on can actually help them to avoid food allergies later on in life.\nB. At this point, your baby will start to develop motor skills and habits, so some of these skills may need replacing. [substeps] Let your baby feed solids at 7 months.\nC. Even though no solids are introduced, moderate to large amounts (approximately 2 liters) are required to give them a solid base. [substeps] You can gradually introduce wheat to their stools as a last resort.\nD. [substeps] You may want to feed them 4 extra batches of liquid food for two feedings, then 3 more servings at 4 months. [title] Add at least half of a hearty greek stir-fry to each individual meal.","prompt":"[header] How to introduce wheat to your baby [title] Introduce solids at 4 months. [step] If your child is transitioning from liquid foods to solids, then you can start feeding them wheat at this time as well. Most children start this transition between 4-7 months.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7139,"error":null,"output":"A"}
{"question_id":"hellaswag:00007516","item_index":231,"attempt":0,"prompt_hash":"85806f53217b6764de26020e4cad4460bafbff73a59265cca737ed84f4c6dd14","question":"[header] How to make the most of your money [title] Pack simple lunches instead of eating at restaurants. [step] Sandwiches and bulk snack items will help your food bill drop. Observe your grocery bills to see what foods or other items (e.g. toilet paper and toothpaste) you buy a lot.\n\nA. Reading labels beforehand will allow you to afford small meals more easily. [substeps] If you eat lunch, visit a supermarket to see what items are included in a convenience section, such as fruit, vegetables and meat.\nB. [substeps] Eat similar foods when you can to save money on gas. [title] Plan lunches that won't make your bills drop.\nC. Buy these in bulk to save money and gas. [title] Try cheaper, healthier alternative snack items at stores like trader joe's.\nD. To avoid combining any purchases that require a certain amount of money into one meal, put a few simple groceries in a lunch or buy the \" walk me down \" bag with a purse that holds all your food, like mini sandwiches and extra bottles. [substeps] Don't be afraid to splurge during grocery shopping.","prompt":"[header] How to make the most of your money [title] Pack simple lunches instead of eating at restaurants. [step] Sandwiches and bulk snack items will help your food bill drop. Observe your grocery bills to see what foods or other items (e.g. toilet paper and toothpaste) you buy a lot.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7137,"error":null,"output":"C"}
{"question_id":"hellaswag:00008355","item_index":232,"attempt":0,"prompt_hash":"1aa039a9d9a761354a5177b876086b067bb8b961033e188cb8abb4eeac4ecf87","question":"[header] How to act if your child has started saying bad words [title] Don't overreact when you hear bad words from your child. [step] Sure, it's not what you want to hear, but the situation isn't doomed. You haven't raised an uncouth monster; there are many avenues through which your child might have been exposed to swearing.\n\nA. Avoid yelling, screaming, chewing gum, and locking doors right out in the open: this can escalate the situation rather than make a huge difference in your child's day. [title] Wait for the urge to curse before speaking.\nB. Many would say that not all swearing is bad, but you need to help your child learn what language is appropriate in what settings. This can take some time, but they'll get there.\nC. Don't expect your child to be instantaneously fluent in a couple words. Instead, give yourself a day or two to adjust your expectations and reactions.\nD. In order to listen well you must be present. Say one, make two, and decide who says the bad thing.","prompt":"[header] How to act if your child has started saying bad words [title] Don't overreact when you hear bad words from your child. [step] Sure, it's not what you want to hear, but the situation isn't doomed. You haven't raised an uncouth monster; there are many avenues through which your child might have been exposed to swearing.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7139,"error":null,"output":"B"}
{"question_id":"hellaswag:00000489","item_index":233,"attempt":0,"prompt_hash":"b48a80476ece89c9fc555cb84b52e712290077ce4eb284818ce5c0377967f2da","question":"A man in a red shirt pours water over his head. Images and video clips of teams playing beach soccer in the sand. video clips\n\nA. , aerial shots of the team's actions, and the players parking their cars.\nB. of different beach soccer teams taking shots at the goal.\nC. of a man slapping his feet on the sand.\nD. of players jumping from a boat into the sand are shown.","prompt":"A man in a red shirt pours water over his head. Images and video clips of teams playing beach soccer in the sand. video clips","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7136,"error":null,"output":"B"}
{"question_id":"hellaswag:00005654","item_index":234,"attempt":0,"prompt_hash":"9a5012a9163d1983cbed1f8d2fd288e40760c8d8132227a2c62a30ed643d7923","question":"[header] How to diagnose aids [title] Watch for mild flu-like symptoms within a month of sexual intercourse. [step] This includes fever, headache, muscle pains, rashes, a sore throat, or swollen glands on the neck. These symptoms usually occur within 2 months of infection.\n\nA. [title] Notice itchy or burning skin during sex. [step] If you experience irritation or sore skin on your genitals, then the infection may have spread.\nB. [title] Check for symptoms of the common cold. [step] Take an over the counter cough suppressant (nsaid).\nC. These are usually the first signs that you may have hiv. [title] Examine your skin for lesions, bumps, or rashes.\nD. [substeps] Immunosuppression is the act of killing any virus or antibiotic that is used for your illness. [title] Evaluate the child's genital area.","prompt":"[header] How to diagnose aids [title] Watch for mild flu-like symptoms within a month of sexual intercourse. [step] This includes fever, headache, muscle pains, rashes, a sore throat, or swollen glands on the neck. These symptoms usually occur within 2 months of infection.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7133,"error":null,"output":"C"}
{"question_id":"hellaswag:00005809","item_index":235,"attempt":0,"prompt_hash":"9a700f502db3fa9a8d0482d08b1dd27cd331e9b108d854e753db29efecdbd361","question":"[header] How to get perfect eyebrows [title] Find the place where your brows should begin. [step] Finding just the right spot for your inner brows to start is key to creating perfect brows, since starting too far in could throw off the balance of your face. To figure out where your inner brows should start, use the following technique : [substeps] As a general starting point, take a pencil or other long, thin instrument and line it up with the inside corner of your eye.\n\nA. It should create a slight curve out the corner. Lower it back into position and not over your eyes.\nB. The place where the instrument overlaps your brow is where it should begin. Mark the spot by making a dot there with an eyeliner pencil.\nC. Measure the length of your inner brow and the width of your outer brow. Make sure it is inside the outer portion of your eye.\nD. Draw a dot just above or just above the inner corner of your eye. Make sure that the lines cross a little bit above or below your eyebrows.","prompt":"[header] How to get perfect eyebrows [title] Find the place where your brows should begin. [step] Finding just the right spot for your inner brows to start is key to creating perfect brows, since starting too far in could throw off the balance of your face. To figure out where your inner brows should start, use the following technique : [substeps] As a general starting point, take a pencil or other long, thin instrument and line it up with the inside corner of your eye.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7137,"error":null,"output":"B"}
{"question_id":"hellaswag:00009033","item_index":236,"attempt":0,"prompt_hash":"282aaf857630735ac6ba49daf9d6a083de52530dfdde9bbd7027da51d975c7a0","question":"[header] How to care for your new baby [title] Support your baby's head. [step] When you pick your baby up, be sure that you are supporting the head and neck. Babies have little muscle development in this area and if they do not have support when lifted, they can become seriously injured.\n\nA. [substeps] Give your baby plenty of support so that they can stand and walk. Encourage them to keep their head bent and not awkwardly relax their neck and shoulders.\nB. Your baby will never be perfect and they could easily injure themselves if they are not supported all the time. [substeps] You will need to support their head and neck by gently placing their head and neck on a small flat surface.\nC. [substeps] Pick up your baby by their hips and arms and place them on your lap so they can't nip at your legs or nose. If your baby is lying on their side, it's probably a good idea to sit near their head to monitor their development and so reduce friction.\nD. Lift them delicately and cradle their head as you do so. [title] Don't worry too much about the soft spot.","prompt":"[header] How to care for your new baby [title] Support your baby's head. [step] When you pick your baby up, be sure that you are supporting the head and neck. Babies have little muscle development in this area and if they do not have support when lifted, they can become seriously injured.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7136,"error":null,"output":"B"}
{"question_id":"hellaswag:00006491","item_index":237,"attempt":0,"prompt_hash":"ad6205bac80752bb79f54a6afa2b461cbbb045a8512b2fca9b5c3c66c8db85d1","question":"[header] How to do a baby model portfolio [title] Take the photos yourself. [step] There is no need to spend a fortune on having professional photos taken of your baby. Legitimate modeling agencies accept and encourage photos that you've taken yourself, since they know that babies grow and change all the time.\n\nA. Just be sure that the photos are high-quality, have good lighting, and show your baby clearly. [substeps] Be sure to take the photos in color, not black and white.\nB. But if you're really serious about developing a professional portfolio, consider taking a few professional photos of your baby model before moving on. This will allow you to tell them that your baby is a child and get their approval.\nC. However, if you don't have them taken professionally, you may want to consider sending a \" mock shoot \" for a less costly portfolio. [substeps] A mock shoot is the formal, based upon your pregnancy and what your baby will look like.\nD. However, by getting photos taken yourself, you will earn extra money. [title] Set up a page on your website.","prompt":"[header] How to do a baby model portfolio [title] Take the photos yourself. [step] There is no need to spend a fortune on having professional photos taken of your baby. Legitimate modeling agencies accept and encourage photos that you've taken yourself, since they know that babies grow and change all the time.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7134,"error":null,"output":"A"}
{"question_id":"hellaswag:00002727","item_index":238,"attempt":0,"prompt_hash":"ff2e28a38ad9046da9a08721b9efcaf441298d164e99429b79c95ddbe5e16242","question":"A man and a woman are standing in a kitchen behind an island with fruits on them. Once the male finishes talking, the woman begins grating a vegetable and cutting an orange in half. she\n\nA. then mixed the three ingredients that she had created together in a cutting block and once it was complete, she proceeded to cut them up and mixed the mixture into it.\nB. then speaks to the camera while measuring and sightening the vegetables, while still speaking to the man.\nC. then squeezes the orange into a clear glass bowl and seasoning it and mixing it together with the previously cut up ingredients.\nD. grabs a large knife and starts slicing the vegetables over and over again, when finished, she dumps old stainless steel in the garbage.","prompt":"A man and a woman are standing in a kitchen behind an island with fruits on them. Once the male finishes talking, the woman begins grating a vegetable and cutting an orange in half. she","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7136,"error":null,"output":"C"}
{"question_id":"hellaswag:00008035","item_index":239,"attempt":0,"prompt_hash":"1586e8ba46c0ba780c8940d456a4094ec12598f31d1ac0e798772a3ba666f30c","question":"[header] How to figure out your life [title] Keep a journal. [step] Document your journey to fulfillment in a journal. Journals provide you with a space to process your thoughts, evaluate your fears, and dream your dreams.\n\nA. Use this space to think about that journey. By having a journal, you can set the direction you take towards your plan for your life.\nB. You may also start to reflect on your day to day lifestyle and talk to friends and family about what it was like to live your life in a journal. [substeps] Write down all of the things you were doing and of the people who impacted your life in your journal.\nC. In fact, they are usually intimate, personable tools that you can use to help you through difficult times. Try doing one journal for every day you've had, then use a journal switch to find a different journal for that day, noting as many times as possible when you would like to \" re-journal.\nD. Writing regularly in a paper journal or maintaining a daily blog will allow you to track your progression to happiness. Writing about your day and your emotions forces you to become more self aware.","prompt":"[header] How to figure out your life [title] Keep a journal. [step] Document your journey to fulfillment in a journal. Journals provide you with a space to process your thoughts, evaluate your fears, and dream your dreams.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7133,"error":null,"output":"D"}
{"question_id":"hellaswag:00001214","item_index":240,"attempt":0,"prompt_hash":"ee7543ad96f1e4658f74007ae8b2c925fa1fa21efa51339248f096f67b5f375d","question":"A boy with a microphone is interviewing a man in sunglasses. they\n\nA. talk back and forth in front of a large boulder.\nB. stand in a line, remaining in the same pose.\nC. talk to the camera some more about the game.\nD. begin with a close up of a flute and the words \" qalwd pull \" join the song.","prompt":"A boy with a microphone is interviewing a man in sunglasses. they","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7135,"error":null,"output":"A"}
{"question_id":"hellaswag:00003274","item_index":241,"attempt":0,"prompt_hash":"ca9cd31afbc9dc44f0b62dbcdc152df4bbb7d2479d4ee1acb5048225087ea50a","question":"[header] How to prevent sweaters from stretching [title] Read the care label. [step] Always read the care label on the inside of your sweaters. You should find out if the sweaters can be washed at home or if they must be dry cleaned.\n\nA. If you don't understand the care label, then look it up on the back of the packaging. [title] Look for straps that are too big.\nB. Find out what washing instructions are included with the care label. Make sure you read the label carefully so you can determine how many washes will be necessary for each garment.\nC. If a sweater is currently dry cleaned but they have stretch marks on them, it might be an incorrect diagnosis. [substeps] Stop buying either new or used sweaters to wash them.\nD. The care label will also tell you how to wash the sweaters, although you should plan on hand washing them just to be safe. [substeps] If your sweater has to be dry cleaned, don't even consider hand washing it at home.","prompt":"[header] How to prevent sweaters from stretching [title] Read the care label. [step] Always read the care label on the inside of your sweaters. You should find out if the sweaters can be washed at home or if they must be dry cleaned.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7134,"error":null,"output":"D"}
{"question_id":"hellaswag:00006262","item_index":242,"attempt":0,"prompt_hash":"421ddcd50e064288267c92e7d1e42680f96f9296ef5355f86645ca4017a50de9","question":"[header] How to turn your rss feed into an android app using appyet [title] Sign up and build your app at appyet.com. [step] This is a free and easy to use platform to build your app. You can also get the pro version for $50 a year so you can run your own ads to make money.\n\nA. You can buy appyet via the web store. Once you complete the sign up, you will receive an email.\nB. [title] Sign up and go to your dashboard. [title] Select the option to add a feed.\nC. [title] Load the account you plan to use for ad advertising. [step] This account will allow you to develop a home page devoted to ad advertising.\nD. Being able to get a lot of potential clients in one day will help with making money big enough. [title] Turn your rss feed into an app with a copy function.","prompt":"[header] How to turn your rss feed into an android app using appyet [title] Sign up and build your app at appyet.com. [step] This is a free and easy to use platform to build your app. You can also get the pro version for $50 a year so you can run your own ads to make money.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7135,"error":null,"output":"B"}
{"question_id":"hellaswag:00007025","item_index":243,"attempt":0,"prompt_hash":"5632e67ddc1a0f652feabe81787c8a0e29eb7f927b760fb2b77a3be0f618c427","question":"[header] How to run a half nelson and far wrist pinning combination [title] Thread one arm under one of your opponents arms while still parallel to the opponent and on top of them. [step] Underhook and cup the shoulder. This arm is going to be the near arm (the arm on the side that you are riding on).\n\nA. [substeps] Spin the arm around and over the opponents arm, pulling them with your upper body (pushing up with your body to knock them off. ) press against the opponent's shoulder with your upper arm to stop the movement and make sure you are right under their arms.\nB. The other arm, your far arm, will go and grab your opponent's wrist on the far side. [title] Rotate your wrist so that your fingers are pointing towards his far shoulder.\nC. [title] Wrap this arm around the opponent's wrists so your as far wrist as possible rests on top of their wrist. [step] If you have a tighter arm, the arm placement will be different.\nD. Now, using this arm you can hold the opponent's arm down by locking the left arm and the right arm under their chin. [title] Lean on the arm in front and push as hard as you can with your left arm in front of you.","prompt":"[header] How to run a half nelson and far wrist pinning combination [title] Thread one arm under one of your opponents arms while still parallel to the opponent and on top of them. [step] Underhook and cup the shoulder. This arm is going to be the near arm (the arm on the side that you are riding on).","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7136,"error":null,"output":"B"}
{"question_id":"hellaswag:00006029","item_index":244,"attempt":0,"prompt_hash":"86b7a68dbf1cb22011683b4ae3464b01e78bf7a291f731794fae486075e457c3","question":"[header] How to recover a gmail password [title] Go to http: // www.gmail.com. [step] Use the link or type the url in a web browser. [substeps] If your email or phone number isn't filled in automatically, type it into the labeled field and click on next.\n\nA. If you accidentally enter your email address in the field, click on next. A confirmation message will appear telling you the password has been successfully deleted.\nB. All of your email addresses will be listed here. [title] Enter the email address of the sender and password associated with that email address.\nC. [title] Click on forgot password? [step] Below the password field. [title] Enter the last password you remember and click on next.\nD. You can also click on \" change password \" to see if any changes are currently required. If you signed in directly from gmail.com, you'll be prompted to enter the email address and password associated with your gmail account.","prompt":"[header] How to recover a gmail password [title] Go to http: // www.gmail.com. [step] Use the link or type the url in a web browser. [substeps] If your email or phone number isn't filled in automatically, type it into the labeled field and click on next.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7135,"error":null,"output":"C"}
{"question_id":"hellaswag:00003021","item_index":245,"attempt":0,"prompt_hash":"6d863183dab66ee4faf9df5920e13674ff9a7dd4d3f6a7251e7d863b7476cc1b","question":"We see a title screen and a lady take a mug and throws the water on the ground and uses the bottom to sharpen her knife. we\n\nA. see a pink bowling lane and a lady teaming with a bunch of dresses.\nB. see the lady sharpening her knife on a ladian sharpener.\nC. then see the lady slice her tomato with ease.\nD. see the lady's knife and she points it between her legs and shows it's deeper then normal from aiming straight down between her legs.","prompt":"We see a title screen and a lady take a mug and throws the water on the ground and uses the bottom to sharpen her knife. we","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7133,"error":null,"output":"C"}
{"question_id":"hellaswag:00007703","item_index":246,"attempt":0,"prompt_hash":"66d65e48eae33cbe8047ececcdf54d34f73ac4eed3a2abb2ba30a11be1238385","question":"[header] How to ask someone to be your study buddy [title] Reflect on your goals. [step] Ask yourself what kind of study buddy might be good for you, what kind of person you would like to study with, and where you would like to study. [substeps] Make note of your strengths and weaknesses.\n\nA. Know what kind of help you can contribute and what kind of help you want to receive. A good study buddy will give help, as well as seek it.\nB. Maybe you have a few good traits that you would like others to have, like an aptitude for math or strong communication skills. Maybe you're a good study buddy who takes notes because he can write down in a notepad what you say.\nC. Then, look out for the best friend or significant other. This is important, as when other people ask what's your biggest flaw, they take advantage of the weaknesses, especially the friend with the most strengths, weaknesses, or skills.\nD. Define your personal strengths as well as your individual weaknesses. [title] Ask people what they think of your study buddy approach.","prompt":"[header] How to ask someone to be your study buddy [title] Reflect on your goals. [step] Ask yourself what kind of study buddy might be good for you, what kind of person you would like to study with, and where you would like to study. [substeps] Make note of your strengths and weaknesses.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7005,"error":null,"output":"D"}
{"question_id":"hellaswag:00004218","item_index":247,"attempt":0,"prompt_hash":"519ba7a3b28dea3c2a8296c9b14f6b66974351622d9e08edb52450583873122b","question":"[header] How to get along with new step siblings [title] Calm down before you approach your step-siblings. [step] If you are angry about something that one of your step-siblings has said or done, it is important to calm yourself down before you approach him or her. Otherwise, you may end up saying or doing something that you do not mean to, or even engaging in physical fighting with your step-sibling.\n\nA. [substeps] Take a moment to take a few deep breaths before approaching your step-siblings. Close your eyes and place your hands on your hips.\nB. You can use relaxation techniques to help yourself calm down, such as : [substeps] Doing deep breathing exercises. Imagining a calm place, such as a sunny beach or a lush forest.\nC. [substeps] Before approaching someone new, try to keep yourself calm. Repeat a calming mantra or phrase to yourself to help relax yourself down.\nD. [substeps] Practice deep breathing, or practice counting to five, before you approach your step-siblings. Through breaths, engage your gut and count to three before you approach.","prompt":"[header] How to get along with new step siblings [title] Calm down before you approach your step-siblings. [step] If you are angry about something that one of your step-siblings has said or done, it is important to calm yourself down before you approach him or her. Otherwise, you may end up saying or doing something that you do not mean to, or even engaging in physical fighting with your step-sibling.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7006,"error":null,"output":"A"}
{"question_id":"hellaswag:00009704","item_index":248,"attempt":0,"prompt_hash":"a5f5562efbc5f6692b123f80e5a80f36d1f2281e0a6efdfaa142631e106f56d6","question":"[header] How to tell a cold from flu in toddlers [title] Look for signs of the common cold. [step] Although runny nose, dry throat, and mild body aches are also signs of the flu, your child might show distinctive signs of a cold. You might notice your toddler sneezing or coughing more.\n\nA. Their eyes may also water if they have a cold. [substeps] It can be hard to determine, but if your child has mild headaches, they may have the common cold.\nB. Some babies are naturally immune from each disease, so he or she might show only mild cold symptoms. [substeps] Give your child a home health alert on the flu if he or she displays unusual symptoms.\nC. Be sure to check for these signs before and during visiting them. [substeps] If your child has a sore throat or runny nose, go see your pediatrician if you suspect the possibility of a flu.\nD. [substeps] For instance, your child might have thick mucus than normal and a strong nose. Something like a stuffy nose or dry mouth might not indicate a cold.","prompt":"[header] How to tell a cold from flu in toddlers [title] Look for signs of the common cold. [step] Although runny nose, dry throat, and mild body aches are also signs of the flu, your child might show distinctive signs of a cold. You might notice your toddler sneezing or coughing more.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7005,"error":null,"output":"A"}
{"question_id":"hellaswag:00006329","item_index":249,"attempt":0,"prompt_hash":"41712785ead19353a4cb132b4bba94a4ca974aa98812a9a41ea07865ced7ad21","question":"[header] How to get into pretrial intervention in williamson county, texas [title] Confirm you don't have prior felony convictions. [step] You won't qualify for pre-trial intervention if you already have felonies on your record. If you don't know your criminal history, talk to your lawyer who can help you get it.\n\nA. You also want to check if you have any convictions against you for criminal offenses. [substeps] Convictions against you include convictions for possession and battery as well as felony convictions.\nB. For example, look for the following : [substeps] Felony deferred adjudications adjudicated felony offenses as a juvenile prior felony convictions [title] Check your prior criminal history. [step] You will be ineligible if you also have other criminal offenses as part of your criminal history.\nC. [substeps] 18 months from the date you were convicted of another offense and over the age of 18 (at which you were sentenced). You have two years to go to your pre-trial intervention hearing.\nD. [substeps] You also can obtain pre-trial intervention through the probate court system and the state court system. You can get selective service from the probate court system if you have been convicted of a felony.","prompt":"[header] How to get into pretrial intervention in williamson county, texas [title] Confirm you don't have prior felony convictions. [step] You won't qualify for pre-trial intervention if you already have felonies on your record. If you don't know your criminal history, talk to your lawyer who can help you get it.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7005,"error":null,"output":"D"}
{"question_id":"hellaswag:00003425","item_index":250,"attempt":0,"prompt_hash":"8b2e4ee4b63122d5518cd84bb3e5b4aaed63dd9d1fec738d8f7c7331abd21ab8","question":"[header] How to deal with your child being expelled from school in australia [title] Know what it is. [step] Expulsion from school means that the child is no longer allowed to attend school at that institution. This is different from suspension where the child is banned from attending school for 1-20 days.\n\nA. Your child will not be allowed to attend school for another 3 days when he is expelled. Unless he is suspended, he will be absent for at least a month.\nB. [substeps] There are different types of expulsions from school. The main two types are expulsion and cancellation of enrolment.\nC. The school must be aware of this so they can correct the inappropriate behaviour. [substeps] The child must also know why they are being expelled.\nD. This period is the shortest period of time for the child and usually only lasts 4 weeks. A child who has been expelled from school typically cannot miss the day of the school year.","prompt":"[header] How to deal with your child being expelled from school in australia [title] Know what it is. [step] Expulsion from school means that the child is no longer allowed to attend school at that institution. This is different from suspension where the child is banned from attending school for 1-20 days.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7005,"error":null,"output":"B"}
{"question_id":"hellaswag:00006707","item_index":251,"attempt":0,"prompt_hash":"4a1bcdc8e0102e2a97e2ee8554b474c5f25d1a878a8df6d75fddcc72500edfa7","question":"[header] How to reduce your sentence for misdemeanors [title] Hire a lawyer. [step] With any criminal matter, whether a felony or a misdemeanor, consulting with a lawyer is a good idea. A lawyer will help you minimize your sentence by negotiating with the prosecution and making valid legal arguments to the court.\n\nA. [substeps] You can also seek help in other immediate situations. Depending on the nature of the crime, you may want to hire a lawyer based on the crime.\nB. [substeps] The easiest way to hire a lawyer is to hire a low-cost investment banker who is adept at representing teenage children and up-to-date on the latest trends. A good investment banker will have powerful experience in contract law, tax law, criminal defense practices, and civil procedure.\nC. A good lawyer may even be able to get the charges against you dropped or reduced before the case ever goes to trial. [substeps] If you cannot afford to hire a lawyer, you can ask to have one appointed to you.\nD. Lawyers will do background checks on you, which will keep you from being prosecuted for an assault or you from being prosecuted for misdemeanors you didn't commit. [title] Use depositions, where you are in court.","prompt":"[header] How to reduce your sentence for misdemeanors [title] Hire a lawyer. [step] With any criminal matter, whether a felony or a misdemeanor, consulting with a lawyer is a good idea. A lawyer will help you minimize your sentence by negotiating with the prosecution and making valid legal arguments to the court.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6873,"error":null,"output":"C"}
{"question_id":"hellaswag:00006841","item_index":252,"attempt":0,"prompt_hash":"415627650d89ece37333e0b43a71d872b2bcc4e0da893986001ecc9ef4debfbf","question":"[header] How to give a manicure [title] Pour some oil into a small bowl. [step] Almond oil, grape seed oil, and sunflower oil are the best. [title] Heat the oil.\n\nA. [step] Add them about 20 seconds before you need to hold them in your hand. [title] Carefully massage your cuticles with your fingertips.\nB. [step] Do not touch the palm of your hand until the oil is hot. Add a few drops of lavender or lemon oil as well, to flavor it.\nC. [step] Put the small bowl of oil in a larger bowl filled with hot water. This will warm the oil evenly and keep it from becoming too hot.\nD. [step] Boil it for 30 minutes or more, and then turn it off. Set the bowl aside in a safe place for later to replace the oil.","prompt":"[header] How to give a manicure [title] Pour some oil into a small bowl. [step] Almond oil, grape seed oil, and sunflower oil are the best. [title] Heat the oil.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6873,"error":null,"output":"C"}
{"question_id":"hellaswag:00004474","item_index":253,"attempt":0,"prompt_hash":"91983a5e2bf111ff0550e5df17e9344acce2174a6de76499f5c2bb514be169a6","question":"[header] How to drive defensively [title] Avoid distractions. [step] Defensive driving means focusing on one thing: driving. Make sure that you're not getting distracted by conversations, the radio, your phone, or anything else.\n\nA. Concentrating on one thing, every few seconds, will ensure that you don't stay on the phone while you're driving. [substeps] This also applies when you're making side-of-road decisions or looking out the window or watching television.\nB. The focus should not be on what others are talking about, but on your driving. [substeps] Do not let your phone get distracted by the radio.\nC. To be as alert as possible, put your phone and other distractions away and turn your radio down. If you're distracted when talking, don't participate in conversations until you reach your destination.\nD. Instead, listen to birds chirping, go for your bike, or even break out your favorite board game. [substeps] Instead of following the instructions on the dashboard, just listen carefully to the instructions as the driver applies them.","prompt":"[header] How to drive defensively [title] Avoid distractions. [step] Defensive driving means focusing on one thing: driving. Make sure that you're not getting distracted by conversations, the radio, your phone, or anything else.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6871,"error":null,"output":"C"}
{"question_id":"hellaswag:00003190","item_index":254,"attempt":0,"prompt_hash":"41611ddf18532c392788ce8736b21130492de52e3ac3a5151582cc9a098cdf88","question":"She is sitting down knitting. She is walking into a building. She sits down and talks with another lady. they all\n\nA. begin knitting sitting next to each other.\nB. go down a snowy hill.\nC. come down together and have a bite at a table.\nD. go into the building.","prompt":"She is sitting down knitting. She is walking into a building. She sits down and talks with another lady. they all","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6871,"error":null,"output":"A"}
{"question_id":"hellaswag:00005081","item_index":255,"attempt":0,"prompt_hash":"ae68f064054edf8221af34f4270a32982a47d53a2a84feaf9f46a6ec040363b2","question":"[header] How to stay cool when you have ms [title] Invest in an air conditioner. [step] To beat the heat, invest in an air conditioner and set it up in your living space. You may get a cheaper, portable model that you can set up in a room as needed.\n\nA. Mix up your house's existing air conditioner settings so that your fans remain constant. [substeps] Air fans should be laid down the same way as air conditioners, keeping them nice and warm in one room and cool in another.\nB. Or you may invest in a permanent air conditioner that cools your home. Weigh the costs against your ability to be comfortable in your home, especially if you live in a climate that often gets hot or warm.\nC. You may also save money to invest in a specialized exhaust fan. [substeps] Make sure the air vent in your home is open, with a fan blowing in from outside.\nD. [title] Be realistic about what your lifestyle will look like while you have ms. [step] It's tough to assess how you are going to live.","prompt":"[header] How to stay cool when you have ms [title] Invest in an air conditioner. [step] To beat the heat, invest in an air conditioner and set it up in your living space. You may get a cheaper, portable model that you can set up in a room as needed.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6871,"error":null,"output":"B"}
{"question_id":"hellaswag:00006754","item_index":256,"attempt":0,"prompt_hash":"dede803afe7b232cbb03bb1d4dbdbc9db018d0a7e922ad6b69cb08f223d90c98","question":"[header] How to prevent hair loss while on accutane [title] Take the lowest effective dose. [step] The more vitamin a you have in your body from accutane, the more likely you are to suffer from adverse side effects, including hair loss. Ask your doctor what the lowest dose is that would be effective for treating your condition.\n\nA. [title] Talk to your doctor about vitamin supplements. [step] Accutane may cause vitamin deficiencies in some patients, so it may be a good idea to have some blood work done.\nB. [substeps] Since vitamin a is supposed to treat split ends, take it as directed. Most doctors recommend 24 doses of vitamin a every day at least 4 hours prior to exercise.\nC. [title] Lubricate the area with a non-stick coating. [step] Don't use stiff-bristled brush to apply the medication.\nD. [substeps] If you try treating your condition with other dosages, just stop taking the highest effective dose. [title] Identify doses, or reduce the dosage.","prompt":"[header] How to prevent hair loss while on accutane [title] Take the lowest effective dose. [step] The more vitamin a you have in your body from accutane, the more likely you are to suffer from adverse side effects, including hair loss. Ask your doctor what the lowest dose is that would be effective for treating your condition.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4889,"error":null,"output":"A"}
{"question_id":"hellaswag:00001688","item_index":257,"attempt":0,"prompt_hash":"3bb1af04b1b4c9c5d5b0fdf66ce8e64f141ebd3f6cd12929df793b7630bb2d24","question":"A woman is standing outside with some white furniture. She begins to point and explain the process to paint her furniture yourself like a diy project. after she\n\nA. is done talking to the camera, she uses various tools to get soft materials while balancing and then wringing them out.\nB. 's finished, she shows how you can hang the furniture, put the paint on it and spread it out in a decorative manner.\nC. 's done trying to paint, she begins putting all all of the pieces together and goes to get more supplies.\nD. shows the colors of the paint that she is planning to use then she grabs some grey paint and paint brush ans begins at the top of the white chair.","prompt":"A woman is standing outside with some white furniture. She begins to point and explain the process to paint her furniture yourself like a diy project. after she","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4895,"error":null,"output":"D"}
{"question_id":"hellaswag:00007940","item_index":258,"attempt":0,"prompt_hash":"e183ff812be9075e0230f215f5801f614d7bffd716a8c5f359ee7758f04c2fdb","question":"[header] How to get the smell of a perm out of your hair [title] Rinse your hair before applying a neutralizer. [step] Ask your hairstylist to thoroughly rinse your hair for 5 minutes. Your stylist will most likely use a clarifying shampoo to help remove any silicone or chemical buildup prior to adding the neutralizer.\n\nA. However, you may have the option of adding a clarifying oil after a shower to clear away any sebum after a shower. You can also use a clarifying lotion or serum instead of these products for a perkier effect.\nB. [substeps] If you have light brown or light blonde hair, skip this step. If your hair is already fairly platinum blonde, you do not need to worry about messing with this.\nC. This will help ensure that your hair is properly cleaned before the chemicals are applied, which may eliminate some of the chemical smell left in your hair by the perm process. [title] Wash your hair with a clarifying shampoo at home.\nD. [substeps] Urine smelly perm hair is quite hard to remove with a pumice stone or a ton of water, so opt for a clarifying shampoo instead. Hot water flushes out all the minerals and other things that you need.","prompt":"[header] How to get the smell of a perm out of your hair [title] Rinse your hair before applying a neutralizer. [step] Ask your hairstylist to thoroughly rinse your hair for 5 minutes. Your stylist will most likely use a clarifying shampoo to help remove any silicone or chemical buildup prior to adding the neutralizer.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4888,"error":null,"output":"C"}
{"question_id":"hellaswag:00003136","item_index":259,"attempt":0,"prompt_hash":"41bdbbfd87c6338e2d23118f8864b385ab13619c12eec93782d639238f3deb03","question":"A large group of girls are seen dancing around a gymnasium holding jump ropes and leads into the girls performing a routine. the girls\n\nA. move all around the area while performing a routine and end by jump roping and raising their arms in the air and leading into more women running onto the stage and cheering with them.\nB. continue jumping all around the stage performing different moves as well as dumping objects on the ground into a bucket and putting them in a vat.\nC. continue the routine all in sync and end by standing together and two high fiving.\nD. dance together while the men stand and watch and watch and end by sliding around in poses while ending with one another sitting at the end.","prompt":"A large group of girls are seen dancing around a gymnasium holding jump ropes and leads into the girls performing a routine. the girls","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4892,"error":null,"output":"A"}
{"question_id":"hellaswag:00003676","item_index":260,"attempt":0,"prompt_hash":"4832dd3cba7613983a94a0fbfc27fdc4541336ece6cba2edf1cfd1f514b0e083","question":"[header] How to buy baby shoes [title] Buy shoes when your baby starts taking steps. [step] Infants don't need shoes, because you're always carrying them everywhere. But once your baby officially starts learning to walk, it's a good idea to have at least one pair of shoes on hand to provide protection from dirt, germs, and dangerous objects.\n\nA. [title] Take steps often, but don't walk in shoes for long periods of time. [step] Babies develop foot arthritis when they are walking, so you can't expect your baby to notice until they get comfortable wearing shoes.\nB. [substeps] It's fine to let your baby practice walking at home without shoes. It might even be easier for them to figure out how to balance out those first wobbly steps when they are barefoot.\nC. [substeps] A child needs shoes around three months old so that they're not worn out in the yard. A toddler needs shoes at six months old to learn to walk.\nD. [substeps] If your baby is unsure about how to walk, refer to a shoe shop, or ask someone in your department if your child can borrow shoes. Consider purchasing shoes from a store that has a treadmill or other machine that your baby may like.","prompt":"[header] How to buy baby shoes [title] Buy shoes when your baby starts taking steps. [step] Infants don't need shoes, because you're always carrying them everywhere. But once your baby officially starts learning to walk, it's a good idea to have at least one pair of shoes on hand to provide protection from dirt, germs, and dangerous objects.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4888,"error":null,"output":"B"}
{"question_id":"hellaswag:00008106","item_index":261,"attempt":0,"prompt_hash":"3d2ce48db697d21f69d61b4c26c24cfc7dee6d1b2295fb24bccb6cf342825fd4","question":"[header] How to style a one piece swimsuit [title] Select a one piece in a bright color or pattern for a vibrant style. [step] Choose fun, funky colors like bright pink, aqua, or lime green. This adds a splash of color to your outfits and beach looks, and you can easily pair this with bright-colored accessories as well.\n\nA. [substeps] You can go with a solid-colored swimsuit in a bright color, or a colorful patterned one piece. For example, look for a bathing suit in hot pink or a suit with hot pink polka dots.\nB. [substeps] Get some beach beads or pretty beach jewelry to help bring out this article of clothing. Hats are also a fun side-step, though you can add decorations such as glitter, sequins, and rhinestones to look more colorful.\nC. [substeps] Some examples of beach themed swimsuit costumes include yoga shorts, exposing shorts, and board shorts. [title] Pick something that is deeper in color or in a lot of space.\nD. You can also mix and match smaller items, like accessories, to create great swimsuits. However, if you're trying to create outfits from multiple items, a blue one piece is probably a better option.","prompt":"[header] How to style a one piece swimsuit [title] Select a one piece in a bright color or pattern for a vibrant style. [step] Choose fun, funky colors like bright pink, aqua, or lime green. This adds a splash of color to your outfits and beach looks, and you can easily pair this with bright-colored accessories as well.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4889,"error":null,"output":"A"}
{"question_id":"hellaswag:00005440","item_index":262,"attempt":0,"prompt_hash":"b2dfa325156e94e38b808fb4c752aa33235dec338a690cd2044411e3f32f29aa","question":"[header] How to make bagel chips [title] Heat the garlic and oil. [step] To help infuse the oil with the garlic flavor, heat them together in a saucepan on low heat for about 15 minutes. Don't let the oil get too hot, as you just want to heat the garlic and oil, not cook them.\n\nA. Don't cut the garlic into chunks-remove it and brown the garlic a bit first. [substeps] You can use natural garlic or a whole clove of minced garlic.\nB. [substeps] Alternatively, you can also puree the garlic with the oil in a blender or food processor. In this case, you can even add spices like rosemary to the mix to give the oil an even more powerful flavor.\nC. [substeps] Vegetable oil works well for this recipe as well. Raw, cooked garlic can be combined with a roux, cooking solution, flour, salt, and water to infuse the oil.\nD. [substeps] If you don't have garlic oil, you can find it in a spice book, in a spice grinder, or in your pantry. [title] Whisk the mayonnaise and lemon pepper.","prompt":"[header] How to make bagel chips [title] Heat the garlic and oil. [step] To help infuse the oil with the garlic flavor, heat them together in a saucepan on low heat for about 15 minutes. Don't let the oil get too hot, as you just want to heat the garlic and oil, not cook them.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4890,"error":null,"output":"B"}
{"question_id":"hellaswag:00005982","item_index":263,"attempt":0,"prompt_hash":"9951a0ac360e8f9d6208c2687f80136e1a9f590a2e637d3c67ab0f59b74892c1","question":"[header] How to diagnose septic arthritis [title] Recognize risk factors. [step] Individuals of any age can develop septic arthritis. However, some people may be more susceptible to this condition.\n\nA. Children, older adults, and illegal drug users are more likely to have septic arthritis. Other risk factors include : [substeps] Having existing joint problems such as gout or lupus.\nB. Schedule an appointment with your healthcare provider to learn more about the risk factors associated with septic arthritis. [substeps] People who use a wheelchair often experience a more frequent flare up.\nC. If you suspect your younger child might be likely to develop septic arthritis, make sure you take him or her to an emergency care facility in the future. Knowing what risk factors your child may face can help you rule out other risk factors as well.\nD. When the disease spreads, it's at risk of getting worse. Potential causes of septic arthritis include cirrhosis, viral or bacterial infections, strokes, and mono-physiological attacks.","prompt":"[header] How to diagnose septic arthritis [title] Recognize risk factors. [step] Individuals of any age can develop septic arthritis. However, some people may be more susceptible to this condition.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4891,"error":null,"output":"B"}
{"question_id":"hellaswag:00008928","item_index":264,"attempt":0,"prompt_hash":"ce8c56acbd1dbfa08db9ade1f5177c627ce8df5ef43347832287eb7d063feb96","question":"[header] How to prevent thrush when breastfeeding [title] Sterilize any feeding instruments. [step] This is particularly important to do if your baby has presented with thrush symptoms. Any instrument that comes into contact with your baby's mouth and your nipple should be sterilized.\n\nA. Wash your hands before and after handling your baby to prevent the spread of the disease. [title] Wash your hands with soap and water before handling you baby.\nB. [substeps] Sterilizing a nursing instrument increases the likelihood of thrush infection. For example, consider sterilizing instruments with tca or.\nC. You can do this by bringing water to a boil and placing the instruments inside for five minutes after every use. The instruments you should sterilize include : [substeps] The teething ring, pacifiers, and the teat of the feeding bottle.\nD. To sterilize a given tool or nipple, wash it thoroughly with soap and water, then dry it with a clean towel. [substeps] You can sterilize an already sterilized tool in the kitchen sink or on the dishwasher.","prompt":"[header] How to prevent thrush when breastfeeding [title] Sterilize any feeding instruments. [step] This is particularly important to do if your baby has presented with thrush symptoms. Any instrument that comes into contact with your baby's mouth and your nipple should be sterilized.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4891,"error":null,"output":"A"}
{"question_id":"hellaswag:00004721","item_index":265,"attempt":0,"prompt_hash":"4b75e7957682d0833074ba58b4ed3ef3f84197b414b60f02e50a0ee423d28bf1","question":"[header] How to make a cardboard dollhouse [title] Find a large shoebox. [step] A shoebox for adult sized shoes would be best. If you can find a shoebox for boots, it will give you even more room to work with.\n\nA. You will also need a long, narrow box to work with, preferably one with a wide opening like a piece of glass at the top. [substeps] Make sure the lid is zip-lock.\nB. It is always nice to have something to place a blanket, pad, pillow case etc inside. [title] Cut the shoebox open with a utility knife.\nC. [title] Cut a 2-inch (5.08 centimeters) wide opening in one corner of the shoebox. [step] The opening should be bigger than you want your dollhouse to be.\nD. [title] Cut the top of the shoebox in half. [step] Use the top of the shoe box to create a roof for your dollhouse.","prompt":"[header] How to make a cardboard dollhouse [title] Find a large shoebox. [step] A shoebox for adult sized shoes would be best. If you can find a shoebox for boots, it will give you even more room to work with.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4887,"error":null,"output":"D"}
{"question_id":"hellaswag:00000146","item_index":266,"attempt":0,"prompt_hash":"6771de4e39eb2a6f6606d78d4ba187531fedf695a18b41c94c937b76b91564a8","question":"People are standing on a hill of snow. people\n\nA. are working inside a building.\nB. are sitting in buckets on a blanket on the hill.\nC. are shoveling snow from the foot of the hill.\nD. are snowboarding down a hill of snow.","prompt":"People are standing on a hill of snow. people","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4890,"error":null,"output":"D"}
{"question_id":"hellaswag:00000245","item_index":267,"attempt":0,"prompt_hash":"b317b240549ca92c66c670e19bb864467b3074370b21756ff85ec41a4142a60d","question":"A man is shown on a stage, playing drums. He beats the cymbals and drums very fast. he\n\nA. then finishes setting up with his sticks.\nB. uses his hands to motion the beats.\nC. continues, making faces as he goes.\nD. continues to drum as several smaller screens appear as they show how to play.","prompt":"A man is shown on a stage, playing drums. He beats the cymbals and drums very fast. he","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4889,"error":null,"output":"D"}
{"question_id":"hellaswag:00003906","item_index":268,"attempt":0,"prompt_hash":"50d3ba9dcbb06996c0c56a46d12160390ac69b7167e5cfeca2e9d801fce3c8a2","question":"[header] How to cut a fringe [title] Determine the kind of fringe you want, depending on your face shape. [step] If you have a long oval face with a wider forehead, try a blunt front fringe. If you have a smaller forehead and rounder face, try a side-swept fringe.\n\nA. The width of your fringe should be the length from one arch of an eyebrow to the other. You can make it slightly wider, but don't make it too narrow.\nB. If you have a shorter face with short hair, choose an updo with curlers. [title] Cut fringe off your forehead if you have very light hair.\nC. [title] Cut away all of the fringe from your face, as that adds volume to your face. [step] If your forehead is wide and you want to end up with a scrunched face, try choosing the side-swept fringe.\nD. But if you have a narrow face with a long face, try a side-swept fringe. [substeps] If you have a wide forehead with short hair, having fringe on the side-swept fringe can work great! [title] Buy a fringe body piece to line the face line.","prompt":"[header] How to cut a fringe [title] Determine the kind of fringe you want, depending on your face shape. [step] If you have a long oval face with a wider forehead, try a blunt front fringe. If you have a smaller forehead and rounder face, try a side-swept fringe.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4889,"error":null,"output":"A"}
{"question_id":"hellaswag:00009914","item_index":269,"attempt":0,"prompt_hash":"cab5ac99cd07c3ca2c02ddd7b9dc1b9436da71e3d1b2d54f37139d27ee821074","question":"[header] How to reset a water heater [title] Disconnect the electricity to the heater. [step] Never tamper with a water heater that's still powered on. Before you begin opening the heater, figure out where it's plugged into the wall and turn off the electricity.\n\nA. [substeps] Most heaters are plugged into a wall with a cable. You may have to unscrew the cable or turn a knob near it to shut off the electricity.\nB. [substeps] The water heater is usually located off from the main unit. [title] Look inside the heater housing to determine if there is a plug-in/battery-operated wall thermometer.\nC. [substeps] Typically there are a series of holes in the wall and a \" charge \" button at the bottom of the water heater. Check the circuit and ensure that the \" charge \" button is turned off.\nD. Water aways often have a hot current running through them. If your running water heater is plugged in, this inputs the electricity into a water supply, whose cord may be plugged in on other appliances.","prompt":"[header] How to reset a water heater [title] Disconnect the electricity to the heater. [step] Never tamper with a water heater that's still powered on. Before you begin opening the heater, figure out where it's plugged into the wall and turn off the electricity.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4887,"error":null,"output":"A"}
{"question_id":"hellaswag:00006071","item_index":270,"attempt":0,"prompt_hash":"d3a92c1c417d381e89025a70563b20e82bf7b126f30a8001ab09ce165503c0e1","question":"[header] How to dress up bobby pins [title] Get some bobby pins, and slide them over the top of a sheet of paper. [step] Thicker paper, such as cardstock or an index card would work best for this, but you can use plain printer paper too. If you plan on painting several bobby pins the same color, consider separating them into small groups based on the color you will be painting them.\n\nA. [substeps] Avoid neon or vibrant colors when using bobby pins. Avoid thicker paper like cardstock, since the color will show through the paper.\nB. [substeps] If you want to create a \" color blocked \" look, get some black bobby pins, and lay down strips of tape over them at various angles. [title] Paint the top of your bobby pins using a single stroke.\nC. [title] Paint four bobby pins at once into each color. [step] If you want your bobby pins one color, try painting two or three in the same color.\nD. [substeps] Use a pencil to draw the shapes you will be painting the bobby pins onto. Tape or stipple glue can work well for this.","prompt":"[header] How to dress up bobby pins [title] Get some bobby pins, and slide them over the top of a sheet of paper. [step] Thicker paper, such as cardstock or an index card would work best for this, but you can use plain printer paper too. If you plan on painting several bobby pins the same color, consider separating them into small groups based on the color you will be painting them.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4887,"error":null,"output":"C"}
{"question_id":"hellaswag:00009058","item_index":271,"attempt":0,"prompt_hash":"ff2fa8ee8a05feb9acb4cd1ac7ec57491c7feb7832ee8faee3517a405a33f7fc","question":"[header] How to jumbo braid [title] Attach a rubber band to the center of the hairpiece. [step] Spread your hairpiece across a flat surface. Place a rubber band at the center where the hair thins.\n\nA. Use your fingers to hold the hair in place. [substeps] The rubber band should be tight enough to stay in place, but loose enough to secure the hair from getting in the way.\nB. Twist the rubber band around the rubber band so that it wraps around the rubber band three times. If the hair thins after looping three times around the rubber band, it is too loose.\nC. Leave some space in the center of the hairpiece to allow the hair to drop easier. [substeps] You can also use a clear elastic band, which is a band shaped to fit tightly on a hairbrush.\nD. Stretch the rubber band into a thin line and wrap it around the thinned hair. Feed one looped end of the rubber band through the other looped end.","prompt":"[header] How to jumbo braid [title] Attach a rubber band to the center of the hairpiece. [step] Spread your hairpiece across a flat surface. Place a rubber band at the center where the hair thins.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4891,"error":null,"output":"D"}
{"question_id":"hellaswag:00001744","item_index":272,"attempt":0,"prompt_hash":"4dfecc571510305f37c0ec775a78a95f86952c94d2b55506dc17106dbbd6d0b0","question":"A man in a weightlifting outfit stands in front of a small group of people and lifts a heavily weighted bar bel from an elevated platform, above his head. A man in a body suit bends before a bar bell with his hands on the bar. the man\n\nA. then lifts the barbell above his head before changing leg stance to better support the weight and finally allowing the bell to drop to the floor from the height of his head and bounce as helpers rush the stage to get the bell.\nB. lifts, puts, and cheers.\nC. performs simple form lifting and lets go immediately.\nD. pumps his gloved hand then spins himself around to bend the bar twice.","prompt":"A man in a weightlifting outfit stands in front of a small group of people and lifts a heavily weighted bar bel from an elevated platform, above his head. A man in a body suit bends before a bar bell with his hands on the bar. the man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4889,"error":null,"output":"A"}
{"question_id":"hellaswag:00002470","item_index":273,"attempt":0,"prompt_hash":"ea76fbabb448b514ea8166786e5463c768a78a0c417077320bfd542a1ca287fe","question":"A young girl stands in the drive way. She starts backing up toward the street. She then runs forward toward the camera. she\n\nA. then does hopscotch on the cement.\nB. then jumps the balance beam.\nC. then runs after a street metrorail parked in the drive way.\nD. runs through a yellow light.","prompt":"A young girl stands in the drive way. She starts backing up toward the street. She then runs forward toward the camera. she","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4890,"error":null,"output":"A"}
{"question_id":"hellaswag:00006734","item_index":274,"attempt":0,"prompt_hash":"44ee4466b12678fdcb5d26ae2a7fd952de90640784511a7b3b1a19680c80c0d0","question":"[header] How to move to hawaii [title] Choose oahu. [step] The island of oahu is the most populated island in the hawaiian chain. Oahu contains honolulu and waikiki beach.\n\nA. It's both a place of commerce and culture. Hawaiian sights and nightlife are high in the hawaiian grand scheme.\nB. North america has eight major lakes in hawaii, with seven lakes written out in scene 2 where residents choose to live. [substeps] Hawaii is 1 of seven islands in the hawaiian chain, and contains the grotons and piles of floodwater, and five lakes.\nC. This island offers the best chances at finding employment as well as great options for entertainment. [substeps] Oahu tends to offer higher salaries than most other islands.\nD. [title] Decide whether you want to move closer to waikiki or end up somewhere more reminiscent of only hawaii. [step] The resorts and amenities given in waikiki receive travelers by bus, postal cab and, thus, by horseback riding.","prompt":"[header] How to move to hawaii [title] Choose oahu. [step] The island of oahu is the most populated island in the hawaiian chain. Oahu contains honolulu and waikiki beach.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4888,"error":null,"output":"C"}
{"question_id":"hellaswag:00005567","item_index":275,"attempt":0,"prompt_hash":"c9efab42c7bae0f348e9cdf28c428263bff0839194d0f140c4c063f7d38ffd55","question":"[header] How to cut knee holes in jeans [title] Put the jeans on and mark the knees where you want the holes. [step] Find a pair of jeans that fit you well, then put them on. Draw a horizontal line across your knee with a pen or a piece.\n\nA. Your lines should be a bit larger than your actual knee holes! [title] Mark where you would like the hole to go. [step] Write down the distance that your line will go to make sure you will be able to keep track of your guidelines for the cutting distance and width.\nB. Over the line, draw a permanent mark where you want the holes to be. Place the jeans on a blanket or set down your pants.\nC. Use this line as a guideline to calculate your stitching distance. [substeps] Try not to forget your measurement after you cut the jeans.\nD. You can do this on just 1 knee, or you can do it on both knees. [substeps] You can create more than 1 slit on each knee.","prompt":"[header] How to cut knee holes in jeans [title] Put the jeans on and mark the knees where you want the holes. [step] Find a pair of jeans that fit you well, then put them on. Draw a horizontal line across your knee with a pen or a piece.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4889,"error":null,"output":"D"}
{"question_id":"hellaswag:00009511","item_index":276,"attempt":0,"prompt_hash":"4bffca2375178b117b4ef7b93f319babc8cff6275cf506d5e07dd9ac51e8a9ba","question":"[header] How to help your child when a friend dies [title] Have an honest discussion with the child. [step] Death can be a confusing and traumatic event for children. It's important to forge a supportive and open atmosphere so that they can begin to cope and to ask any questions that they may have.\n\nA. [substeps] For example, you might ask the child, \" how do you cope if your friend dies? \" or \" why do you feel the way you do? \" [title] Ask the child to talk. [step] Give them some encouragement by asking them questions about their loved one.\nB. [substeps] Cover any situations with a \" flesh and blood \" theme. Let the child know that you are there if they need to talk.\nC. If one of your child's friends has died, make yourself emotionally available. [substeps] How you discuss death will depend a lot on your child's age.\nD. Open communication is the best way to help your child adjust and become better at relating to those around them. [substeps] Avoid giving overly judgmental, but compassionate statements like, \" you've got to see that you can find the grave take it easy.","prompt":"[header] How to help your child when a friend dies [title] Have an honest discussion with the child. [step] Death can be a confusing and traumatic event for children. It's important to forge a supportive and open atmosphere so that they can begin to cope and to ask any questions that they may have.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4889,"error":null,"output":"C"}
{"question_id":"hellaswag:00008725","item_index":277,"attempt":0,"prompt_hash":"d554cf4400a34e7af888617fb0172aedf78a7d990fab53d8c5f0e38e27b8ac31","question":"[header] How to be a twitter celebrity [title] Tweet like a famous blogger. [step] Twitter is a micro-blogging tool; it's basically blogging in 280 characters or less in each tweet. Generally, the same guidelines for being a good blogger apply for being a good tweeter.\n\nA. [substeps] Tweet \" @widbettydisputing91.net \" in an ad. Twitter posts cite the original twitter post, the tweet, and any other source that relates back to that post.\nB. Opinions should be shared and there should be a diverse range of opinions about twitter status, content, and behaviors. [substeps] Magazines and sites are better sources of information.\nC. [substeps] Twitter needs facts and statistics to promote your products and services. Check out blogs and other websites before attempting the self-talk exercise: http: // twitter.com/tweets/0-constant [title] Use hashtags in your content.\nD. You can't expect a blog with only three posts on random topics to attract many subscribers, can you? [substeps] Choose your topic. If you want to be famous on twitter, don't take their question (\" what are you doing right now? \") literally.","prompt":"[header] How to be a twitter celebrity [title] Tweet like a famous blogger. [step] Twitter is a micro-blogging tool; it's basically blogging in 280 characters or less in each tweet. Generally, the same guidelines for being a good blogger apply for being a good tweeter.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4890,"error":null,"output":"D"}
{"question_id":"hellaswag:00004409","item_index":278,"attempt":0,"prompt_hash":"205a994966b53f5db0f288a7fc0efcef9003c5161b53b93b7b56de67ad6a0bc4","question":"[header] How to stop forcing yourself to act okay [title] Stop worrying about what others think. [step] To be yourself and stop pretending, you have to stop analyzing how you come across to others. Not focusing on how you're acting allows you to instead focus on what you're doing, which is when people are typically themselves.\n\nA. [substeps] If you see any people who come off as overly confident or arrogant, remind yourself that they're not what they're looking for. Put yourself in their shoes, and ask yourself questions like, \" am i acting okay? \" to set clear boundaries and respect yourself.\nB. Not constantly seeking approval from others by pretending to feel good when you don't, allows you to be yourself, which is usually more enjoyable than putting on a charade. [substeps] Whenever you catch yourself worrying about others' opinions of you, quickly evaluate your situation.\nC. Here are some things you can do to give up the mentality that you're the total opposite of how the person looks at you : [substeps] Realize that you are human. At one point you might judge a person for being crazy, which may be hard, but this is okay.\nD. One study showed that replacing negative thoughts with positive ones can help \" force \" the behavior. [substeps] Example: maybe i am running late to class due to a coworker who isn't impressed with my performance.","prompt":"[header] How to stop forcing yourself to act okay [title] Stop worrying about what others think. [step] To be yourself and stop pretending, you have to stop analyzing how you come across to others. Not focusing on how you're acting allows you to instead focus on what you're doing, which is when people are typically themselves.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4381,"error":null,"output":"B"}
{"question_id":"hellaswag:00005290","item_index":279,"attempt":0,"prompt_hash":"85d54e8a203c311e0c108a4a08c41e45af9b39f238691d71d0e9a3b7eaa43200","question":"[header] How to stop enjoying carbs [title] Say \" no \" to white bread. [step] Bread is the most common simple carbohydrate we consume and the one that is easiest to remove. [substeps] Try lettuce wraps.\n\nA. Lettuce wraps are the easiest toppings to remove, but they are equally effective at reducing the number of times you feel nauseous. Try olive oil and a fresh lemon and/or cucumber.\nB. Many different greens are a great source of carbohydrates, even when you are in the habit of eating a lot of them. Try tomato in a stir fry, fish salad, or brownies in a lasagna, for example.\nC. Wrap the top of a head of lettuce cut in quarters--simply roll one or two slices into a ball or catch them with a quick, powerful hand as the wrapper changes from red to black. Two other bread options to treat are giant intestine wraps.\nD. Romaine lettuce works well to substitute for flat breads, while butter lettuce and radicchio make an excellent substitute for sandwich bread. Simple add sandwich filling to the lettuce and fold over and around, or add a second piece of lettuce to the top.","prompt":"[header] How to stop enjoying carbs [title] Say \" no \" to white bread. [step] Bread is the most common simple carbohydrate we consume and the one that is easiest to remove. [substeps] Try lettuce wraps.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4380,"error":null,"output":"D"}
{"question_id":"hellaswag:00003321","item_index":280,"attempt":0,"prompt_hash":"6e80caa399eacc3bef5c9f5c52e61ec04206c3a38d67726e982b0f7901058f46","question":"[header] How to ignore bullies [title] Avoid giving a bully the opportunity to bully you. [step] Most bullies will look for easy opportunities to victimize someone, or \" easy targets \". Try to avoid being an easy target by being aware of the bully's movements when you are in the same room with them and focus on keeping to yourself.\n\nA. [substeps] For example, choose a table further away from the bully at lunch, or a seat in class that is not within eyesight of the bully. It's also important to keep an eye on your belongings and avoid giving the bully an opportunity to bully you.\nB. Since bullies often target others because they value solitude and comfort, you will not be able to do this well. [substeps] If you find yourself feeling tempted to call up the bully or cry in front of them, do not.\nC. [substeps] Don't immediately cover up or hide when in class. Hiding in your locker or a study corner can't be very effective.\nD. [substeps] Walk carefully, keep your head high and look straight ahead of you and avoid eye contact with strangers. Ignore every over-exaggeration they make, no matter how silly it is.","prompt":"[header] How to ignore bullies [title] Avoid giving a bully the opportunity to bully you. [step] Most bullies will look for easy opportunities to victimize someone, or \" easy targets \". Try to avoid being an easy target by being aware of the bully's movements when you are in the same room with them and focus on keeping to yourself.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4381,"error":null,"output":"A"}
{"question_id":"hellaswag:00002544","item_index":281,"attempt":0,"prompt_hash":"5ef6ac48e525e3b75ad27007c8698270f90bdc7ffb6d9102a70c33e3e4c26b69","question":"One of the gymnast dressed in red and white leotards is performing gymnastics on bars as his coach watches him. He does a hand stand and then does a front flip as he swings on the bars. he\n\nA. does another hand stand and a front flip as he twists his body.\nB. dismounts after landing, still waving his arms in the air.\nC. does a handstand as he's climbed to the top and lands on a mat with his mascot on top of him.\nD. starts doing a back flip on his left leg.","prompt":"One of the gymnast dressed in red and white leotards is performing gymnastics on bars as his coach watches him. He does a hand stand and then does a front flip as he swings on the bars. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4381,"error":null,"output":"A"}
{"question_id":"hellaswag:00002987","item_index":282,"attempt":0,"prompt_hash":"ee3a3a9107b28379bdc575a7d70090593b5e9a2e0b598b15218408fdc2b66b74","question":"The group members enjoy themselves in the tubes are the go downhill. They share some pictures of their tubing adventure. the snowy mountain slopes\n\nA. , landscapes and their onlookers.\nB. climb into sight and a group comes together on the slope.\nC. have several other tubers enjoying themselves as they go down the slopes.\nD. and waterfall is shown.","prompt":"The group members enjoy themselves in the tubes are the go downhill. They share some pictures of their tubing adventure. the snowy mountain slopes","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4380,"error":null,"output":"C"}
{"question_id":"hellaswag:00006313","item_index":283,"attempt":0,"prompt_hash":"b52060cee7d61734afdfb0f75566d0b6e22be3bff4d1786adbf9a599bc9196dc","question":"[header] How to take care of your skin while on accutane [title] Take cool, brief showers. [step] Accutane is notorious for drying out the skin. Cool water showers will dry your face out less than hot showers and can keep the side effects at bay.\n\nA. In turn, a cool shower can minimize the damage caused by the accutane. [substeps] If you're in a rush, or notice a burning sensation, try to take some time out of the shower before you plan to do something you'll regret.\nB. Keeping the shower short likewise ensures your skin is not stripped of essential oils and does not dry out. Long showers and baths dry out your skin, so try to limit them to 5 to 10 minutes.\nC. It also helps ensure that you don't damage your skin. [substeps] People with accutane will find a shower too warm and then splash lukewarm water on their face for 7-10 minutes.\nD. If you plan to make any kind of major decision, you can skip the hot showers. They can provide more protection from the sun and strip the upper half of your skin of its natural oils.","prompt":"[header] How to take care of your skin while on accutane [title] Take cool, brief showers. [step] Accutane is notorious for drying out the skin. Cool water showers will dry your face out less than hot showers and can keep the side effects at bay.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3634,"error":null,"output":"B"}
{"question_id":"hellaswag:00005827","item_index":284,"attempt":0,"prompt_hash":"e4220f3f6f1f6bdf9cf5b5de8b046bc9cb460762635770ae9a56cb40e839cefb","question":"[header] How to cuff jeans [title] Look at pictures of different cuff styles. [step] In case you hadn't noticed, cuffs are everywhere, in many different forms. Do research in fashion magazines or online, or simply notice how other people are cuffing their jeans.\n\nA. [title] Find a cuff that works for you. [step] Factors to consider include the width and weight of your jeans, the type of footwear you expect to pair with them, and whether you want a cuff that adds length to your silhouette.\nB. Get some ideas for cuffs that go with many outfits and styles. [substeps] For example, a sweater or dress is almost as likely to be cuff-resistant as a baseball hat and army style briefcase.\nC. Think about alternate ways of wearing the cuff. [title] Don't make yourself look faint.\nD. [substeps] Also, try to see what's currently fashionable. Don't just see it in the pictures.","prompt":"[header] How to cuff jeans [title] Look at pictures of different cuff styles. [step] In case you hadn't noticed, cuffs are everywhere, in many different forms. Do research in fashion magazines or online, or simply notice how other people are cuffing their jeans.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3633,"error":null,"output":"A"}
{"question_id":"hellaswag:00004618","item_index":285,"attempt":0,"prompt_hash":"b3fe92f875c28712fffd0548554f2712d9447dd393618fa0ad44de4bb1d04cb0","question":"[header] How to deal with opinionated people [title] Figure out if you need to confront this behavior. [step] Interacting with opinionated people is unavoidable, so, you'll have to pick your battles wisely to make sure that you're not engaging in verbal warfare when such people cross your path. Not every annoying comment needs to be dealt with.\n\nA. [title] Reward the person for his or her perceived rudeness. [step] There will come a time when you should keep unliked comments fresh in your head as the person may take offense or start to resent them.\nB. These folks will become extremely annoyed if you are left to your own devices to vent the frustrations they throw at you. Once you've determined the issue, you'll have options.\nC. Also, it may not be in your best interest to try to confront a person. [substeps] Is this an opinion worth spending your time disputing? Some opinions may be annoying but is pointless to argue.\nD. Try to come to a resolution with these \" ums \" of people before you decide to keep bothering them. [substeps] If you're feeling down, excuse yourself for a moment and speak to a trusted close friend, family member, or neighbor.","prompt":"[header] How to deal with opinionated people [title] Figure out if you need to confront this behavior. [step] Interacting with opinionated people is unavoidable, so, you'll have to pick your battles wisely to make sure that you're not engaging in verbal warfare when such people cross your path. Not every annoying comment needs to be dealt with.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3634,"error":null,"output":"C"}
{"question_id":"hellaswag:00004699","item_index":286,"attempt":0,"prompt_hash":"5f353fcf1ddd6eae89f22b119b5c036732e5d5caf3242cbe770f2e42b8c8ead8","question":"[header] How to do festival makeup [title] Prep your face. [step] Before you apply your eye makeup, you will need to create a base. This will help to ensure that your eye makeup lasts all day long.\n\nA. [substeps] Draw a round, square, or diagonal line around your eye. Trace the circle around your eye, then draw a rectangular strip across the middle.\nB. [substeps] Start with a clean, moisturized face. Start by cleansing your face with a gentle cleanser, then apply a light moisturizer to your face and neck to reduce the appearance of fine lines.\nC. Before applying any eyeshadow, pick and measure a single eyeshadow that is slightly larger than the eye you want to blend. Apply the eyeshadow to the tip of each eye and sweep a blending brush across them.\nD. Wearing makeup first thing in the morning is not always an option, but you can do it in the evening. [substeps] Shower to keep your skin clean and moisturized.","prompt":"[header] How to do festival makeup [title] Prep your face. [step] Before you apply your eye makeup, you will need to create a base. This will help to ensure that your eye makeup lasts all day long.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":3634,"error":null,"output":"B"}
{"question_id":"hellaswag:00001373","item_index":287,"attempt":0,"prompt_hash":"5a105422c8ee0bf01804b03aac81d31805b483d444c9838450c1272eb2877a1f","question":"A boy rinses his face with water in a bathroom sink. the boys\n\nA. wash his face with their hands after they brush his teeth.\nB. laugh and joke together in the bathroom.\nC. turns on the faucet.\nD. stand under the stream of water battling each other.","prompt":"A boy rinses his face with water in a bathroom sink. the boys","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":3633,"error":null,"output":"C"}
{"question_id":"hellaswag:00009701","item_index":288,"attempt":0,"prompt_hash":"3069272029b01b08616d319c6a5adb386aeb44887f7adec0d64f7bcd743387fb","question":"[header] How to lose 20 pounds [title] Record your current eating patterns. [step] Take the first week to analyze your current diet habits. To do this, start a food diary.\n\nA. This contains a daily log of how many calories you eat each day. Write down when you eat lunch or when you visit the gym for your regular gym classes.\nB. [substeps] A food diary means record your meals, snacks, activities, and daily routine, before working on weight loss. In this diary, include any foods you are avoiding and whether or not you are following a strict low-calorie or low-carb lifestyle program.\nC. You should note the things you eat and do while you are struggling to lose weight, the times you exercise, and also the types of calories you cook and drink. Getting a journal will be a great help with this.\nD. Record everything that you eat and drink for a week, be sure to include snacks and portion sizes. [substeps] As you record your food and drink intake, also record your mood.","prompt":"[header] How to lose 20 pounds [title] Record your current eating patterns. [step] Take the first week to analyze your current diet habits. To do this, start a food diary.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5615,"error":null,"output":"D"}
{"question_id":"hellaswag:00007247","item_index":289,"attempt":0,"prompt_hash":"07bd33aba8a9037fddb15fde719a1f9c4d36aa21ef2bd0e8f836111f9f1ee825","question":"[header] How to apply pancake makeup [title] Wet your sponge. [step] Pancake makeup needs to be activated with water, and should be applied with a dense sponge. The great thing about pancake makeup is that you can control the amount of coverage you get by adding more or less water (less water means less diluted makeup and heavier coverage).\n\nA. [substeps] For regular coverage, soak your sponge with water and then squeeze out the excess so the sponge is damp. For lighter coverage, soak the sponge and then give it a gentle squeeze so it's not dripping wet.\nB. This means your makeup will be waterproof for a while, which means the matte makeup is the most consistent when you apply it. [substeps] If you don't have a sponge, you can just use a wet sponge.\nC. [substeps] When wetting the sponge, just blot the butter where you wish your pancake makeup to be. It's usually easiest to apply with the sponge when wiping your skin clean, as you won't have to worry about mistakes in little areas.\nD. [substeps] You can squeeze the sponge directly into the butter, juice, or other container with the appropriate amount of pressure. Alternatively, you can use a bowl or dish large enough to hold the makeup of your choice and mix and stir the two ingredients together.","prompt":"[header] How to apply pancake makeup [title] Wet your sponge. [step] Pancake makeup needs to be activated with water, and should be applied with a dense sponge. The great thing about pancake makeup is that you can control the amount of coverage you get by adding more or less water (less water means less diluted makeup and heavier coverage).","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5616,"error":null,"output":"A"}
{"question_id":"hellaswag:00000867","item_index":290,"attempt":0,"prompt_hash":"5388262ff99a3451e6b8631aebadb54196be34dc203cc9fd5e8742d004adbadb","question":"The person pushes the mower along the edge of the grass and throughout the rest of the grass. the man\n\nA. stops and turns around to hand off the frisbee.\nB. then grabs a weed wacker and cuts the weeds along the edge.\nC. puts the machine on the grass and picks up some leaves to put on the lawn mower.\nD. rolls down the lawn mowing path making turns and turns around.","prompt":"The person pushes the mower along the edge of the grass and throughout the rest of the grass. the man","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5620,"error":null,"output":"B"}
{"question_id":"hellaswag:00000364","item_index":291,"attempt":0,"prompt_hash":"7665887940209b563a95e7cfa2e9ca38dd96fff5729ac577d45daefb01c8ad4e","question":"Two men perform a flip move on a playground that leads to one being dropped on the ground. the camera\n\nA. follows the men down the path.\nB. zooms in on the fallen man while they all laugh together.\nC. rolls off and turns around on itself in slow motion.\nD. follows one man to the ground, but is blocked by a garbage can and one man surrounds him with objects.","prompt":"Two men perform a flip move on a playground that leads to one being dropped on the ground. the camera","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5618,"error":null,"output":"B"}
{"question_id":"hellaswag:00007654","item_index":292,"attempt":0,"prompt_hash":"6b12a7e41e5a9ad15e172ef8e29e5ca868d76a8aadd0d341c0e4235f0f7a00e9","question":"[header] How to make soup [title] Make a single vegetable soup. [step] Although other flavors are usually added to a soup focused on a single vegetable, such a soup is predominantly the flavor of the main vegetable used. Examples include : [substeps] Tomato soup or creamy tomato soup pumpkin soup cauliflower soup carrot soup cabbage soup sweet corn soup.\n\nA. [title] Cool the soup in the stockpot. [step] Place the stockpot of soup back in the pot and turn it on its burner.\nB. [title] Make vegetable-based soups which contain a variety of vegetables and other ingredients. [step] Soups which mix veggies can come up with an enormous range of delicious flavors.\nC. [substeps] In other words, if you are planning on using a nonindian main vegetable, you should focus on making a single vegetable soup. [title] Check that your vegetables are ready for boiling.\nD. Cauliflower soup is typically served warm or cold due to the sweetness of the vegetables. Chicken soup, on the other hand, is served cold due to the sweetness of the vegetables.","prompt":"[header] How to make soup [title] Make a single vegetable soup. [step] Although other flavors are usually added to a soup focused on a single vegetable, such a soup is predominantly the flavor of the main vegetable used. Examples include : [substeps] Tomato soup or creamy tomato soup pumpkin soup cauliflower soup carrot soup cabbage soup sweet corn soup.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5614,"error":null,"output":"D"}
{"question_id":"hellaswag:00007093","item_index":293,"attempt":0,"prompt_hash":"cf7f2dbf4433e40619145e2151248c84e1bb70df4087e82e3f64e07c06ccd218","question":"[header] How to lace shoes [title] Put a shoe in front of you with the toe facing away from you. [step] Starting from the two frontmost opposite facing holes, insert each end of the shoelace from the inside. Make sure both sides of the remaining laces are equal.\n\nA. Pull the two laces through one more hole from the inside of the shoe before tossing it away. [substeps] This step is not required if you use a shoe that is not elbow-length or has neck-length.\nB. [title] Working in a diagonal direction, insert the right end of the shoelace into the next left hole (from the top). [step] Alternate method: you can insert the lace pointing from the front of the shoe inward (instead of going from the inside pointing outward), to give a more neat appearance.\nC. Slide both laces through one the middle hole and into the opposite hole. [title] Slide one end of the lace through the open-toe hole and back through the other one.\nD. [substeps] The laces will appear to be together now. With this position, once you've put the shoes on, you need to make sure you only have one position for both laces.","prompt":"[header] How to lace shoes [title] Put a shoe in front of you with the toe facing away from you. [step] Starting from the two frontmost opposite facing holes, insert each end of the shoelace from the inside. Make sure both sides of the remaining laces are equal.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5614,"error":null,"output":"B"}
{"question_id":"hellaswag:00004562","item_index":294,"attempt":0,"prompt_hash":"356051cb7dd83486fe7769e6d961db34b1ad2685d286275aeabd53a4300917e5","question":"[header] How to treat feline retinal disease [title] Watch your cat's movements in dark areas. [step] The retina is made up of structures called photoreceptors-rods and cones-that allow your cat to see light and color. With pra, the rods and cones start breaking down.\n\nA. Often, lumps or blobs are the result of an obsession with bright white centers from aging retinal skin. You may notice a particularly large pile of this fungus a few hours after a quit session.\nB. Each class of mesh refracts light into one different area. The easiest way to rid your cat of photoreceptors is to place them in narrow, hiding spots, such as the casing housing of an outdoor refrigerator.\nC. Without the rods, your cat would not be able to see very well in dark areas, like a hallway. You may notice your cat avoiding dark areas of your home.\nD. This dislodges an existing layer of color from your cat's skin. Only a part of the brain can see this formation.","prompt":"[header] How to treat feline retinal disease [title] Watch your cat's movements in dark areas. [step] The retina is made up of structures called photoreceptors-rods and cones-that allow your cat to see light and color. With pra, the rods and cones start breaking down.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5615,"error":null,"output":"C"}
{"question_id":"hellaswag:00007724","item_index":295,"attempt":0,"prompt_hash":"52c4172877eae07027fad859218cda3c751c48bed226e0166e4fa91e39ebfd82","question":"[header] How to get freckle tattoos [title] Look at reference photos. [step] Check out photos of people who have natural freckles online. Use the photos as a guide when deciding how freckled you would like your look to be.\n\nA. [title] Pick the type of shade to choose. [step] Juvenile freckles have a lot of small dents which you shouldn't squish out.\nB. [substeps] Also use the photos as a guide to determine how light or dark you would like your freckles to be. Choose a couple different looks to test out.\nC. Unlined nails, jagged brows, and freckles should all be considered before you buy these tattoos. [substeps] Your tattoos and freckles should not be too obvious.\nD. [substeps] Other things to look for include bold, bright colors, dark spots, light freckles, etc. If you find someone who has freckles, and they aren't wearing clothes you typically wouldn't mind avoiding, this person may be a good match for you.","prompt":"[header] How to get freckle tattoos [title] Look at reference photos. [step] Check out photos of people who have natural freckles online. Use the photos as a guide when deciding how freckled you would like your look to be.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5614,"error":null,"output":"B"}
{"question_id":"hellaswag:00005266","item_index":296,"attempt":0,"prompt_hash":"f6a18fff43e4fe7e01a7735d35d49690c2bdbce2f1304c6bc77a6524c2d03d63","question":"[header] How to diversify sandwich fillings [title] Make cheese sandwiches more interesting. [step] While cheese sandwiches are an easy mainstay for lunchboxes, they can be made a lot more interesting than just simple cheese. Here are some ideas : [substeps] Change the cheese.\n\nA. If you've been eating only one kind of cheese, branch out and try other cheese varieties, especially those with stronger flavors, and cheese types you've never tried before. Red pepper, cress, or radish to mild cheese for a piquant taste.\nB. Go for cheddar, cognac, or mojito. Put a corner of cheese on your sandwich for variation.\nC. The types of cheeses are the best for sandwiches. Cheddar, rumansey, and gruyere are both great to use because they both fit into your sandwiches.\nD. For example, change the red cheese for one sandwich in bold and replace the white cheese with cheddar instead. That is, you could eat everything in a sandwich you've made.","prompt":"[header] How to diversify sandwich fillings [title] Make cheese sandwiches more interesting. [step] While cheese sandwiches are an easy mainstay for lunchboxes, they can be made a lot more interesting than just simple cheese. Here are some ideas : [substeps] Change the cheese.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5614,"error":null,"output":"A"}
{"question_id":"hellaswag:00005277","item_index":297,"attempt":0,"prompt_hash":"ca533dbe1ffd6cb46845332a5e01bd5cec63871379f1f5d670a6e185a8e6aee5","question":"[header] How to make a natural fruit perfume [title] Assemble the ingredients. [step] For this recipe, you will need the following items : 1 organic cinnamon stick , 25 organic cardamom pods , 15 organic cloves , 1 organic vanilla bean , 1 fresh organic orange peel (zest only) and 8 ounces of vodka. [substeps] Cut the vanilla bean into small pieces using a sharp knife.\n\nA. Zest the orange peel by rubbing the peel over a cheese grater. When you reach the white portion of the rind, stop grating and move on to another portion of the peel.\nB. This will make room for the cloves and the vanilla bean. You can soften the beans with tea.\nC. Use the juice from 1 orange to use as a thick syrup. You may adjust the amount of vodka in the syrup a couple of weeks before making the perfume for a step-by-step decision.\nD. Roll the slices into a ball using a rolling pin, then have them dip in vodka to coat them. Place the pieces in a jar with a lid, and shake hard to coat the balls.","prompt":"[header] How to make a natural fruit perfume [title] Assemble the ingredients. [step] For this recipe, you will need the following items : 1 organic cinnamon stick , 25 organic cardamom pods , 15 organic cloves , 1 organic vanilla bean , 1 fresh organic orange peel (zest only) and 8 ounces of vodka. [substeps] Cut the vanilla bean into small pieces using a sharp knife.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5614,"error":null,"output":"A"}
{"question_id":"hellaswag:00007739","item_index":298,"attempt":0,"prompt_hash":"11a7964d04aef18516f09465f11ec30acbed942ff5b890cb2fe9ff2597000dcf","question":"[header] How to get rid of problematic skin [title] Drink eight glasses of water per day as this is a natural cleanser which will cleanse your skin out and make it look more radiant. [title] Get eight to nine hours of sleep 7 days a week because this is your skins time to repair itself. [title] Wash your pillowcase weekly to stop transferred bacteria clogging your pores.\n\nA. [title] Put a small dollop of salicylic acid on your lips now and then of course. [step] Use it on your face and leave it on for a week before going out and doing something else.\nB. [title] Cut your facial with a special razor. [step] Make a relatively short cut across your face with a razor to cut the hair off your face.\nC. [title] Sleep on your back, even with a clean pillowcase, bacteria can still lurk. [title] Cleanse your skin twice a day, once in the morning and at night.\nD. [step] Wash in the morning do not just use your pillowcase, wash in the evenings. [title] Make sure you wipe off on a daily basis.","prompt":"[header] How to get rid of problematic skin [title] Drink eight glasses of water per day as this is a natural cleanser which will cleanse your skin out and make it look more radiant. [title] Get eight to nine hours of sleep 7 days a week because this is your skins time to repair itself. [title] Wash your pillowcase weekly to stop transferred bacteria clogging your pores.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5615,"error":null,"output":"C"}
{"question_id":"hellaswag:00005587","item_index":299,"attempt":0,"prompt_hash":"3089feff35929b04d996d20dbcf1d9ae311193c3dd51e4f4e703b501e3d62f5e","question":"[header] How to decorate your yard for halloween with or without money [title] Clean up your yard. [step] Rake leaves, pick up trash, animal litter, and mow the lawn. [title] Now, gather your supplies.\n\nA. [step] The following items will make it easier to treat your yard like a haunted house for halloween : [substeps] Plus you will have treats and snacks. Leftover trash bags and diy costumes.\nB. [step] Stock up on newspaper that's cool to dry. Spray paint if needed to cover the area of your yard you wish to decorate.\nC. [step] You'll need the following items : [substeps] Masking tape. Masking tape is a can that is hung over your mailbox and helps remove odors from the surrounding environment.\nD. [step] You may want to take a trip to the dollar store, walmart or any other store. Have, pumpkins, skeletons, signs, tombstones, coffins.","prompt":"[header] How to decorate your yard for halloween with or without money [title] Clean up your yard. [step] Rake leaves, pick up trash, animal litter, and mow the lawn. [title] Now, gather your supplies.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5616,"error":null,"output":"D"}
{"question_id":"hellaswag:00008763","item_index":300,"attempt":0,"prompt_hash":"efe36d511c99e0abbb472ff951cccc065af5202763763b3beb05fccd2fe6c1c5","question":"[header] How to talk to your kids about homosexuality [title] Expect to have several conversations. [step] Lgbt issues are complex, so you can't cram all of them into one 20-minute conversation. You will probably have quite a few conversations over their childhood, and that's normal and okay.\n\nA. However, your children may want to talk about things they may not normally talk about with their peers or close friends. You may want to remember each discussion for 10 minutes, then go get them some special treats to discuss.\nB. [substeps] If your child seems disinterested, stop talking. They won't learn much if they've tuned you out.\nC. Explain to them that homosexuality is normal, and that most importantly, they understand it. Don't feel like the conversation is a one-sided conversation-it's a whole conversation.\nD. Remember that lgbt communications are based around relationships. You and your partner may not communicate much, and that's ok.","prompt":"[header] How to talk to your kids about homosexuality [title] Expect to have several conversations. [step] Lgbt issues are complex, so you can't cram all of them into one 20-minute conversation. You will probably have quite a few conversations over their childhood, and that's normal and okay.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5615,"error":null,"output":"C"}
{"question_id":"hellaswag:00009799","item_index":301,"attempt":0,"prompt_hash":"236576c9d26926474c202facae29304bd9c7e360c1868807e6325597e61c3136","question":"[header] How to choose a hat [title] Consider your hat needs. [step] Do you want a hat to wear every day? For a special event? Are you buying the hat for fashion needs? For warmth or sun protection? What seasons do you expect to wear the hat in? Knowing this in advance can narrow your options from the start. [title] Familiarize yourself with the broad range of hat styles.\n\nA. [step] Not every hat today fits according to the broad spectrum of styles and trends, so speak to a knowledgeable librarian. They can help you pick a hat for your special event.\nB. [step] Will you want a traditional hat, an opera hat, a taper hat, a fedora, etc. ? These are general-issued hats to represent most men's hats.\nC. [step] Different hats are appropriate to different types of situations and events. [substeps] It is a good idea to travel to a place where you have a variety of hats on display.\nD. [step] There are all kinds of hats in the world, from casual to kentucky derby. Look through fashion magazines, catalogues, or online to get some ideas.","prompt":"[header] How to choose a hat [title] Consider your hat needs. [step] Do you want a hat to wear every day? For a special event? Are you buying the hat for fashion needs? For warmth or sun protection? What seasons do you expect to wear the hat in? Knowing this in advance can narrow your options from the start. [title] Familiarize yourself with the broad range of hat styles.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5616,"error":null,"output":"C"}
{"question_id":"hellaswag:00005229","item_index":302,"attempt":0,"prompt_hash":"8a50b085e407d404de9a70458d87d364e8e71f059cebc421fa159a44cefac64e","question":"[header] How to make vegan chicken nuggets [title] Put the chickpeas in a food processor and pulse until no whole beans are visible. [step] If you don't have a food processor available, a fork or potato masher will work just fine. [title] Add ingredients to a medium-sized bowl.\n\nA. [step] For the creamiest chicken nuggets, start by adding a tablespoon of raw soy sauce, a pinch of salt, and a teaspoon of flour, which will keep the nuggets moist until you finish preparing them. [title] Smash the cubes of chickpeas with a fork.\nB. [step] It's a good idea to have about 10 tablespoons of leftover chickpeas per batch of chicken. [title] Add 2 cups of water and sauté the chickpeas.\nC. [step] Add in textured vegetable protein, vital wheat gluten, mashed chickpeas, soy sauce, mustard, spices, water, and agave mix together with a spoon. [title] Place the chickpea mixture on a clean, flat surface.\nD. [step] Use a 1 ounce package to about 3/4 cup. [title] Stir everything together with a fork until creamy.","prompt":"[header] How to make vegan chicken nuggets [title] Put the chickpeas in a food processor and pulse until no whole beans are visible. [step] If you don't have a food processor available, a fork or potato masher will work just fine. [title] Add ingredients to a medium-sized bowl.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5615,"error":null,"output":"C"}
{"question_id":"hellaswag:00004484","item_index":303,"attempt":0,"prompt_hash":"e49f16039848e10efd24c699b6f165909a8d8b23042f3351fd6c2e06ed2efe49","question":"[header] How to screen print a t shirt [title] Choose a picture to print on to your t-shirt. [step] The picture you choose should be small enough to fit in the embroidery hoop. Choose a simple picture without many small details so there is less chance of the ink bleeding.\n\nA. [title] Place the transparent material in the embroidery hoop securely and place the hoop on top of the picture you wish to transfer to the t-shirt. [step] The material should be laying directly against the top of the picture.\nB. [title] Choose a design if you want to print in specific colors or textures. [step] For example, if your shirt is plain with a grid or an image that is in bold colors, try printing in brown or black.\nC. [substeps] Printing on plain t-shirts is best, as it's easy to warp and can produce an obvious stain if the image overlaps in size or details are taken off. If you're printing for a school project, you may want to go with a picture from the school newspaper with a large design for the print.\nD. [substeps] If you are printing on plain fabric, choose a picture with a contrasting color to create a pattern. If you are printing patterned white shirts, you could print with wild colors like polka dots, circles, or chevrons.","prompt":"[header] How to screen print a t shirt [title] Choose a picture to print on to your t-shirt. [step] The picture you choose should be small enough to fit in the embroidery hoop. Choose a simple picture without many small details so there is less chance of the ink bleeding.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5615,"error":null,"output":"A"}
{"question_id":"hellaswag:00005329","item_index":304,"attempt":0,"prompt_hash":"77960f773383dae6016d252ea6d56bb8f9b3a650e35ee45d3408bffa885a8f97","question":"[header] How to get spiral curls [title] Wash and condition your hair. [step] If you have naturally curly or wavy hair, apply a leave-in conditioner. If you have straight hair, skip the conditioner and use a detangling spray.\n\nA. Getting your roots wet will help manage frizz. [substeps] If you're having flyaways or waves, you want to apply the right amount of conditioner.\nB. Part your hair on the side that isn't straight (the part in the arch). Squeeze the conditioner from your hair, then use a styling cream to hold your hair in place.\nC. [title] Comb out your hair and apply a product. [step] If your hair is naturally curly, use a curl-defining gel.\nD. [substeps] Leave-in conditioners are gentler than air dry. Most hair products prehydrate and hydrate your hair.","prompt":"[header] How to get spiral curls [title] Wash and condition your hair. [step] If you have naturally curly or wavy hair, apply a leave-in conditioner. If you have straight hair, skip the conditioner and use a detangling spray.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6161,"error":null,"output":"C"}
{"question_id":"hellaswag:00007970","item_index":305,"attempt":0,"prompt_hash":"099e4facbecdfa0e4cb7e1a74afa2fa4d23ca10b61cc9be55c2374276a3dddd5","question":"[header] How to discuss finances together in a marriage [title] Broach the subject casually with your spouse. [step] The time to start talking about merging your finances is before the wedding, but at least 40% of couples avoid doing so. [substeps] Start the conversation with your action items first.\n\nA. For example, you could bring up your future plans with your spouse in anticipation of the wedding, or you could say something like, \" i have been thinking about retiring to san francisco and am looking for an apartment right now. I thought you'd be able to help me out.\nB. For example, if you're trying to add a certain amount of money to the prenuptial agreement, talk it over before you talk about how you're going to divide the funds. At the same time, try not to bring up the financial aspects of separation or child support.\nC. This could mean starting off by talking to your spouse about your desire to look at your own credit score as you prepare to buy a house and suggest that he or she does the same. Say something like \" have you checked your credit report lately? I've been wanting to get a good picture of my financial standing.\nD. For example, you could say something like, \" do you want to pay off the mortgage on your car if we live really close? \" depending on your situation, you might want to explain why you might want to split joint accounts. You could say, \" i think separating accounts would be a good idea.","prompt":"[header] How to discuss finances together in a marriage [title] Broach the subject casually with your spouse. [step] The time to start talking about merging your finances is before the wedding, but at least 40% of couples avoid doing so. [substeps] Start the conversation with your action items first.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5615,"error":null,"output":"C"}
{"question_id":"hellaswag:00009910","item_index":306,"attempt":0,"prompt_hash":"6504078a0075b94e0e22f44d6dc0e2eb066729b144408b98f9f673f52791dc75","question":"[header] How to keep rabbit bottles from freezing [title] Fill the water bottle with warm water. [step] Do not use scalding hot water. This will cause it to take it longer for the water to freeze.\n\nA. [substeps] Warm the water bottle in the microwave for 30 seconds at 75 degrees fahrenheit (75 degrees celsius) to thaw the water. Cold water can also kill rabbits with lesions such as stingers.\nB. You can use either filtered or distilled water. Use the method described above if your rabbit will drink cooler water.\nC. However, you will probably still need to change the water several times during the course of the day. [substeps] When doing this touch the stream of water with your hand.\nD. Use enough water to cover the entire bottle. You should water only at below boiling point.","prompt":"[header] How to keep rabbit bottles from freezing [title] Fill the water bottle with warm water. [step] Do not use scalding hot water. This will cause it to take it longer for the water to freeze.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5615,"error":null,"output":"C"}
{"question_id":"hellaswag:00006583","item_index":307,"attempt":0,"prompt_hash":"6df5b9b10c7ba522c44c7de366ad36e860a265e1e8eb4f8af3c00fa8755039e0","question":"[header] How to organize your free time with your family [title] Write down what you and your family members did last week. [step] Use a blank sheet of paper or a weekly calendar. Include your work schedule, family activities, time spent watching tv, and your daily commute.\n\nA. A good starting point is to write down all that you all do each week and how much you do each week in your free time. A great starting point is to focus on accomplishing tasks like : [substeps] Completing your chores regularly connecting to your family.\nB. [substeps] Put a rubber band over the paper for a little extra precaution. Check off tasks before you lay them out on paper.\nC. [substeps] Divide the information by your personal schedule. For example, you might write down after-dinner plans, dance plans, and band-related events.\nD. Be brutally honest with your schedule, even if you don't like what you see. Examine it to identify what you and your family spend the most time doing.","prompt":"[header] How to organize your free time with your family [title] Write down what you and your family members did last week. [step] Use a blank sheet of paper or a weekly calendar. Include your work schedule, family activities, time spent watching tv, and your daily commute.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5616,"error":null,"output":"D"}
{"question_id":"hellaswag:00008134","item_index":308,"attempt":0,"prompt_hash":"f08596634c2463765302f0ab76964013d557f92ac0d010550b3fcfcb0e698ef7","question":"[header] How to wear a cargo skirt [title] Choose a skirt length based on your body and preference. [step] Cargo skirts usually hit at or above the knee. Shorter or petite girls might choose a shorter, above-the-knees style to lengthen their legs, while those with wider hips or thighs may want to avoid a skirt that cuts off at the widest part of the thighs.\n\nA. When in doubt, go with a knee-length cut, which is flattering for all body types. [substeps] Maxi-length cargo skirts are also available.\nB. Top it off with a color that brings out the color of your own skin. [substeps] Choose a long black or grey skirt if you have pale skin.\nC. [substeps] If your body is proportional to your skirt length, or you are curvy and it measures above your knee circumference, then your skirt should be of this size. It should not spike up to the knee when worn.\nD. [substeps] Consider how wide you want your skirt to be. Skirts that fall just below the knee should be longer than skirts that hit below the knee.","prompt":"[header] How to wear a cargo skirt [title] Choose a skirt length based on your body and preference. [step] Cargo skirts usually hit at or above the knee. Shorter or petite girls might choose a shorter, above-the-knees style to lengthen their legs, while those with wider hips or thighs may want to avoid a skirt that cuts off at the widest part of the thighs.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5615,"error":null,"output":"A"}
{"question_id":"hellaswag:00008118","item_index":309,"attempt":0,"prompt_hash":"97ea186c38896c6a7b0f0aa70fba83073f15f1b7128338c8dd254f048dc022a7","question":"[header] How to get a hard boiled egg out of a bottle [title] Gather the necessary materials. [step] To get the egg out of the bottle using this method, you will need baking soda, vinegar, and a plate. This method uses a chemical reaction to increase the air pressure inside the bottle.\n\nA. You will need : [substeps] A glass jar or a baking dish large enough to grip the egg in it. You will need vinegar and either a wire mixing bowl or a wire whisk.\nB. The increased air pressure will push the egg out of the bottle. [substeps] You need about 3 tablespoons (44 ml) of vinegar and 2 tablespoons (14.\nC. Look in the refrigerator or freezer for an egg and vinegar mixture (usually labeled \" eggshell, egg white, vinegar, or baking soda \"). [title] Transfer the egg and vinegar mixture to a clean plate.\nD. It can be done with water or a vinegar solution. [substeps] You can buy baking soda at your local grocery store.","prompt":"[header] How to get a hard boiled egg out of a bottle [title] Gather the necessary materials. [step] To get the egg out of the bottle using this method, you will need baking soda, vinegar, and a plate. This method uses a chemical reaction to increase the air pressure inside the bottle.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6159,"error":null,"output":"B"}
{"question_id":"hellaswag:00007001","item_index":310,"attempt":0,"prompt_hash":"74d38c4ba931017ac4736bec37b16a5c97700126037c2671b2d9bafe85218011","question":"[header] How to make axe bombs [title] Buy a bottle of axe and the rest of your supplies (listed below). [step] Any axe aerosol spray will work, since it is flammable. [substeps] The bomb will work great with any fragrance.\n\nA. But the stink will last a while, so pick the fragrance that you think is most offensive. Make sure you get a long neck lighter.\nB. This spray works well for cooking and dishwashing. [title] Put the axe in the bottle and slide the lid closed.\nC. Any fruit extract will work, but light green apricots are more suitable. [title] Cut the axe heads into stick shapes.\nD. Of course, it will not work with an especially sweet blend, however. [title] Put on safety clothing if you plan to use it for cooking.","prompt":"[header] How to make axe bombs [title] Buy a bottle of axe and the rest of your supplies (listed below). [step] Any axe aerosol spray will work, since it is flammable. [substeps] The bomb will work great with any fragrance.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5793,"error":null,"output":"A"}
{"question_id":"hellaswag:00002433","item_index":311,"attempt":0,"prompt_hash":"c7b6e43abb891562d047b56ebef20c4dcb3b120bf03ec10a810433a3a94c2ef2","question":"A man in a black hat is holding a violin. He begins to play the violin. he\n\nA. stops playing, and sings.\nB. pauses and puts the violin down.\nC. puts the violin down and continues talking.\nD. sings a song about how he plays it.","prompt":"A man in a black hat is holding a violin. He begins to play the violin. he","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5792,"error":null,"output":"B"}
{"question_id":"hellaswag:00009559","item_index":312,"attempt":0,"prompt_hash":"176eefa00c83159b8db83c09077384006d4523609a17ec455a606b3a6f12360e","question":"[header] How to create a steam facial with essential oils [title] Open your sinuses and treat colds. [step] Several essential oils can reduce symptoms of the common cold and open your sinus passages. Add a total of 3 to 7 drops of peppermint, eucalyptus, or oregano essential oils.\n\nA. If you're feeling congested, oregano can treat sinus infection. Peppermint essential oils can treat headaches caused by congestion and eucalyptus can treat the congestion itself.\nB. Mix the oils together with 1 teaspoon of honey , 1 teaspoon (4.9 ml) of baking soda and 1 teaspoon (4.9 ml) of lemon juice. Place the bowl of tea bags over your sinuses for five minutes.\nC. Pour it into your pores (with a dropper) and let it sit for 20 minutes. [substeps] Do not wash your sinuses before applying the essential oils.\nD. Measure out 3 drops of each into a glass of water for five to 10 minutes. [substeps] If you have a particularly nasty nose, the steam will help reduce the pungent smell of the sore.","prompt":"[header] How to create a steam facial with essential oils [title] Open your sinuses and treat colds. [step] Several essential oils can reduce symptoms of the common cold and open your sinus passages. Add a total of 3 to 7 drops of peppermint, eucalyptus, or oregano essential oils.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5791,"error":null,"output":"A"}
{"question_id":"hellaswag:00009614","item_index":313,"attempt":0,"prompt_hash":"fe88047c5f12084176162870ce3ce21f608fb66d97adce841d619b3fb8837f48","question":"[header] How to clean colored leather [title] Vacuum or shake off loose debris. [step] Use a furniture attachment to remove all extra dirt and debris from your furniture. Be very gentle when vacuuming so you don't scratch the leather.\n\nA. Also, be mindful of the decal. If the decal gets damaged, then your leather may become burnt or discolored.\nB. It's more safe to vacuum cushions, fan covers, or coffee tables than to completely clean your leather. [title] Choose a vacuum cleaner that will prevent dust from getting into your upholstery.\nC. Shake dirt into a bucket or wide container. Shake it until you've collected all the dust and debris.\nD. Brush dust and debris off of other leather items, like purses and gloves. [title] Make a basic cleaning solution with mild soap and water.","prompt":"[header] How to clean colored leather [title] Vacuum or shake off loose debris. [step] Use a furniture attachment to remove all extra dirt and debris from your furniture. Be very gentle when vacuuming so you don't scratch the leather.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5792,"error":null,"output":"D"}
{"question_id":"hellaswag:00003519","item_index":314,"attempt":0,"prompt_hash":"e71f0dcf9543eb412452e50d8e69c04b5af19f60a6043a960828e068ce29e363","question":"[header] How to care for snakes [title] Buy a terrarium that will suit the snake. [step] A terrarium is a glass enclosure for snakes. It looks like a fish tank with no water in it.\n\nA. Make sure it is specifically made for holding snakes as they are very good escape artists and will likely escape without a secure lid. You may need to purchase a long or tall terrarium depending on the breed of snake you get.\nB. Terrariums can be the best choice as they allow the snake access to the water (and substrate) of the tank, and provide a temporary head back to underwater. If your tank requires a lid, look for something that is about 8 or 10 feet (3.7 or 15.\nC. [substeps] Reptile aquariums, or reptile aquariums, are more natural in appearance, than aquatic aquariums. They are air-tight aquariums that are kept on neutral: a tank has to pass air currents at the top but water can drain out.\nD. It has to be big enough for the snake to enter and exit the tank. Terrariums are portable and require less space, allowing for larger snakes to fit comfortably in.","prompt":"[header] How to care for snakes [title] Buy a terrarium that will suit the snake. [step] A terrarium is a glass enclosure for snakes. It looks like a fish tank with no water in it.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5792,"error":null,"output":"A"}
{"question_id":"hellaswag:00000503","item_index":315,"attempt":0,"prompt_hash":"d8cc3cc85e2e22307f69c53560808e9c843242bb652dfaeb9ead6b935e63c5e8","question":"A person is seen riding on roller blades looking into a trash can. People look around and watch the man skate. he\n\nA. continues skating around on the skate boards.\nB. continues speaking to others while still moving faster and faster.\nC. skates down a set of stairs and off into the distance.\nD. skates on the street and goes took the stairs.","prompt":"A person is seen riding on roller blades looking into a trash can. People look around and watch the man skate. he","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6170,"error":null,"output":"C"}
{"question_id":"hellaswag:00001091","item_index":316,"attempt":0,"prompt_hash":"5fb3c4d420d73bac2be7f84531f771fd410358c7720bc000983a880518341368","question":"Woman is sitting in a bed talking to the camera showing a contac lenses case. then she\n\nA. mixes solution into the case and put liner in the container and put the black lid on top.\nB. is standing with trunks sitting next to her using a assist tool.\nC. show the contact lens case.\nD. take off hr glasses and show how to put the contact on the eye, she grab it with the index finger and put it in the eye.","prompt":"Woman is sitting in a bed talking to the camera showing a contac lenses case. then she","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6170,"error":null,"output":"C"}
{"question_id":"hellaswag:00003957","item_index":317,"attempt":0,"prompt_hash":"1b38c851919707fd772557e0ae28e27c53b6e698b28d5e83fdc8c0ce33493fd4","question":"[header] How to make your butt bigger fast [title] Do squats. [step] Stand with your feet shoulder-width apart. Bend your knees and slowly lower your butt towards the ground, as if you're going to sit in a chair.\n\nA. Keep your back straight and your chest up. Repeat eight times at a time until your butt is the size of a golf ball.\nB. Focus on getting your knees as close to your butt as possible in a parallel arc with your feet. After you are close enough to the floor, slowly increase the amount of pressure your body is applying on the arches of your feet, keeping your knees flexed.\nC. Keeping your hands on the floor, lower your butt and keep your knees bent, pointing your toes. Hold this position for a few seconds, breathing in and out, then slowly rise back to a standing position.\nD. Go as low as you can without bringing your butt below knee level. Hold for a few seconds at the lowest point, then slowly raise yourself back up.","prompt":"[header] How to make your butt bigger fast [title] Do squats. [step] Stand with your feet shoulder-width apart. Bend your knees and slowly lower your butt towards the ground, as if you're going to sit in a chair.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6170,"error":null,"output":"D"}
{"question_id":"hellaswag:00002132","item_index":318,"attempt":0,"prompt_hash":"ccf5db5dbf11a1bbca749d231a3aa108f3965f349bcc116bce89fd6c9917d972","question":"A weather map is shown for a second, then two news women sitting at a desk. Several people are shown warming up for a race. A gun is fired, and the race begins. we\n\nA. see one great race, following by children on horseback riding and winning.\nB. go back to the women in the news room, talking.\nC. then see a game of volleyball being played as matches are played.\nD. see still images of teams playing field hockey.","prompt":"A weather map is shown for a second, then two news women sitting at a desk. Several people are shown warming up for a race. A gun is fired, and the race begins. we","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6170,"error":null,"output":"B"}
{"question_id":"hellaswag:00005628","item_index":319,"attempt":0,"prompt_hash":"bed428a89a81ba4ca288dc373716a35dd0b6e6ebc96c34b8987130211cadaa57","question":"[header] How to be street smart [title] Avoid wearing headphones while you are out. [step] While it may be enjoyable to listen to music while you walk around, it is best to stay as alert as possible. Not being able to hear your surroundings will make you vulnerable to accidents or attacks.\n\nA. If you absolutely have to listen to music or other audio while you're out, use only one earphone or play it at a very low volume. [title] Do not look at your phone while you are walking.\nB. It's also too risky to be distracted while you are trying to focus on a particular task. Try to avoid leaving your headphones on until you have no choice but to move around a bit to get the most out of your listening.\nC. Always work at it: make sure to put noise pollution out of the equation, and then put it at a lower volume (like going up the ac, heating up a window, using an air conditioning unit). [substeps] If possible, wear headphones when you are out, and press down on the volume button.\nD. Putting headphones on makes it even more difficult to slip in and out of your location. [substeps] You might think it a subtle way to keep yourself out, but if you are in a public place or in situations where you can slip in and out unnoticed, you can still avoid mishap risks by wearing headphones.","prompt":"[header] How to be street smart [title] Avoid wearing headphones while you are out. [step] While it may be enjoyable to listen to music while you walk around, it is best to stay as alert as possible. Not being able to hear your surroundings will make you vulnerable to accidents or attacks.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6170,"error":null,"output":"A"}
{"question_id":"hellaswag:00008868","item_index":320,"attempt":0,"prompt_hash":"eba3e11ee32fa07fd5a91f0fdcd1594fd221e25fa6018a027729c37bf804cdf7","question":"[header] How to protect your hair while sleeping [title] Use moisturizing products after showering. [step] If you shower or wash your hair at night, apply a serum or leave-in conditioner while your hair is still damp to smooth and strengthen your locks. This is a good first step to prevent damage while sleeping.\n\nA. [substeps] When you're trying to maintain healthy hair, reapply a serum after your shower or bath to hydrate and moisturize your locks. [title] Use a silk pillowcase if you want to remove stubborn locks.\nB. A good formula for this is hydroquinone. [title] Remove makeup and clothing as soon as you wake up.\nC. Do not rub or apply petroleum-based products around the hairline. In areas with bumps, gently massage the area with a massage serum.\nD. [title] Apply an oil treatment or hair mask if you plan wash your hair in the morning. [step] Apply an overnight hair mask or oil treatment, cover your hair with a scarf or shower cap, and wash it out when you wake up.","prompt":"[header] How to protect your hair while sleeping [title] Use moisturizing products after showering. [step] If you shower or wash your hair at night, apply a serum or leave-in conditioner while your hair is still damp to smooth and strengthen your locks. This is a good first step to prevent damage while sleeping.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6193,"error":null,"output":"D"}
{"question_id":"hellaswag:00004612","item_index":321,"attempt":0,"prompt_hash":"b2c6551bbf95da54b6c51316402b2fe6f6cc860f42f78d83bb30f188d73bc824","question":"[header] How to hold a straight razor [title] Hold the straight razor in your dominant hand. [step] It will be easier to grip and use the straight razor using the hand you normally shave with. If you usually shave with your non-dominant hand, use that hand instead.\n\nA. [title] Maintain a straight razor by holding it vertically. [step] Keep your chin on the top of the curved side of the blade.\nB. [substeps] Because the straight razor is curved and sharp, keeping the two metal tip forks isolated in your hand should be harder than if you used your non-dominant hand. [title] Consider using the razor when shaving your legs.\nC. [title] Release your finger from the razor and grip the sharp edge. [step] Keep your hand relaxed as you move it toward the blade, as though you're making a fist.\nD. [title] Place your index, middle, and ring finger on the back of the blade. [step] Position your fingers so they're at the base of the blade.","prompt":"[header] How to hold a straight razor [title] Hold the straight razor in your dominant hand. [step] It will be easier to grip and use the straight razor using the hand you normally shave with. If you usually shave with your non-dominant hand, use that hand instead.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6192,"error":null,"output":"D"}
{"question_id":"hellaswag:00006943","item_index":322,"attempt":0,"prompt_hash":"4d336f205dbe13593c6a0306b7ca90810a903d3a77cfdc9392242b7b0651854a","question":"[header] How to apply basic stage makeup (for women ) [title] Start with a layer of foundation, applied with a latex sponge. [step] You want to achieve a nice even coverage. For lighter skin tones, choose a foundation a few shades darker than your natural tone as this will help combat the bleaching effect of stage lights.\n\nA. For bolder skin tones, use dark eyeshadow and apply the foundation while still maintaining the stain. [substeps] Use a sun-kissed shade for the most even coverage.\nB. [title] Before setting the foundation, make sure that any creases around the eyes have been worked out with the sponge. [title] Shake out a neutral setting powder onto either your palm or a tissue.\nC. For hair, experiment with bringing up to 2 shades lighter, then using concealer, hair dye and mascara. [substeps] The foundation should be thin, concealable and waterproof.\nD. This can vary depending on your skin and your age, but it should never take much less than two sessions per week to get an even coverage shade perfect. [substeps] Apply your makeup with small, downward strokes.","prompt":"[header] How to apply basic stage makeup (for women ) [title] Start with a layer of foundation, applied with a latex sponge. [step] You want to achieve a nice even coverage. For lighter skin tones, choose a foundation a few shades darker than your natural tone as this will help combat the bleaching effect of stage lights.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6193,"error":null,"output":"B"}
{"question_id":"hellaswag:00005076","item_index":323,"attempt":0,"prompt_hash":"18138fa9142c6ad80a58ca54c47820d85ca35bb735a28d0fc4e66312f5e412e7","question":"[header] How to treat a swollen knee [title] Compare your affected knee with your other knee. [step] Look for puffiness around the kneecap or around the sides of the knee. Comparing your two knees is a good way to check to see if there is swelling or redness and to see if is something wrong.\n\nA. [substeps] There might also be swelling at the back of your knee. This could signal a baker's cyst, which is when extra fluid is pushed into tissue behind your knee.\nB. [substeps] Look for tenderness that is accompanied by soft, tearing or ripping skin. You may see redness and swelling along the edges or along the seams.\nC. [substeps] Measure your affected knee and then compare it to your other knee. It is generally the same as comparing two knee surgeries but it is also slightly different.\nD. Swelling-swelling is an odd, but clearly, very serious swelling, which you may have noticed either before or after surgery. Swelling in an inflamed knee usually occurs after surgery and won't require immediate medical attention.","prompt":"[header] How to treat a swollen knee [title] Compare your affected knee with your other knee. [step] Look for puffiness around the kneecap or around the sides of the knee. Comparing your two knees is a good way to check to see if there is swelling or redness and to see if is something wrong.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6192,"error":null,"output":"A"}
{"question_id":"hellaswag:00001675","item_index":324,"attempt":0,"prompt_hash":"d115a5766524fdc1f851dce94065ca6eb7b6424cbacac91a3a5936ac07521349","question":"A man wearing a suit is sitting behind a desk with papers in front of him, a blue screen behind him, and he's talking into the camera while words scroll at the bottom. video clips\n\nA. of hockey players are displayed in taking turns and shifting across the screen without really using a device.\nB. are shown of the home he's renovating and steps are shown on the stairs as well as the roof, the roof, teams lined up, and people wearing signs talking.\nC. are shown of people performing flips on the same floor where people pass, and shows them performing several flips before the man walks back the way he came.\nD. of hockey players on a rink and people in the stands begin to play while words are still scrolling on the bottom, and at times a man is shown talking to the camera with blue words on the bottom that say \" amateur ice hockey league crowned with gala game against past legends \".","prompt":"A man wearing a suit is sitting behind a desk with papers in front of him, a blue screen behind him, and he's talking into the camera while words scroll at the bottom. video clips","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6201,"error":null,"output":"C"}
{"question_id":"hellaswag:00008429","item_index":325,"attempt":0,"prompt_hash":"00fb1d03b024165d5c406b49d9022a3560d90e63a14943176c4b474e38a1b3ef","question":"[header] How to treat diarrhea in ferrets [title] Familiarize yourself with what is normal for your ferret. [step] A ferret's normal stool should be soft, but still formed (rather than runny). Check your ferret's cage or litter box from time to time and observe what its healthy stool looks like.\n\nA. If you notice anything unusual, such as stools that are runny, green, or foul-smelling, contact your vet. [title] Take your ferret to the vet if you notice diarrhea.\nB. Ferrets' stools shouldn't be watery, frothy, or thick. If they are, they may be sick.\nC. [substeps] Healthy stools should be neutral color, i.e. solid green, and with no small flecks of green or other textures.\nD. If you spot something like loose feces or urine, that's normal. The thinner your ferret's stool, the better.","prompt":"[header] How to treat diarrhea in ferrets [title] Familiarize yourself with what is normal for your ferret. [step] A ferret's normal stool should be soft, but still formed (rather than runny). Check your ferret's cage or litter box from time to time and observe what its healthy stool looks like.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6192,"error":null,"output":"A"}
{"question_id":"hellaswag:00009923","item_index":326,"attempt":0,"prompt_hash":"5d3501926ab3250b6f1379ab3619dd8f43e2538248e1e151b255a7ee12d9c080","question":"[header] How to apply natural makeup for brown eyes [title] Stick to earth tones and other natural shades. [step] Keep in mind that certain colors will look natural on some people while looking obvious on others. Stick to shades close to your skin, lip, and hair colors.\n\nA. [substeps] If your eyebrows and eyelashes are black or very dark brown, then black eyeliner and mascara will look natural on you. For red hair, try out auburn or brown mascaras to find one that suits you best.\nB. [substeps] Your brows should match up slightly with your lip color, and you should not wear an uneven lip color when you apply other makeup as it may seem less natural or appealing. Be aware that skin tones can come in many shades, so if you want to wear your eyebrows naturally they need to look natural.\nC. If you feel like it may get your hands dirty, go somewhere lighter or cream-based. [substeps] It doesn't need to be professional if you want to go rock bottom and work your way up from there.\nD. [substeps] If you go for a grayish shade, splash some on your skin and thin it out. If you go for a pastel color, put it on your lips and blend it with a brush.","prompt":"[header] How to apply natural makeup for brown eyes [title] Stick to earth tones and other natural shades. [step] Keep in mind that certain colors will look natural on some people while looking obvious on others. Stick to shades close to your skin, lip, and hair colors.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6195,"error":null,"output":"A"}
{"question_id":"hellaswag:00005539","item_index":327,"attempt":0,"prompt_hash":"835e325fdea7653a638851725527ebfb2d38b692ec7214b248d9108390cc6633","question":"[header] How to prevent weevils from invading your pantry [title] Learn to identify weevils. [step] The best way to prevent weevils from getting into your home is to avoid buying food that's infested with them. To do this, you have to be able to identify weevils.\n\nA. Find a large, open room (such as a spare bedroom) with lots of windows. Wear gloves and identify the weevils by sight.\nB. [substeps] Both rice and granary weevils have divided bodies made up of a head, thorax, and abdomen. Each species ranges from one-eighth inch (3.2 mm) to three-sixteenth inches (4.8 mm) long.\nC. You can do this by finding out who they are and committing to certain food practices. [substeps] In order to find out if a particular food is a weevils problem, you may have to make specific lifestyle changes to identify whether or not your dish is a weevils problem.\nD. If you don't know how to identify weevils, look through your pantry and see if your pantry has any around your home. [substeps] Look for infested food that has plants, fresh fruits, and dairy.","prompt":"[header] How to prevent weevils from invading your pantry [title] Learn to identify weevils. [step] The best way to prevent weevils from getting into your home is to avoid buying food that's infested with them. To do this, you have to be able to identify weevils.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6194,"error":null,"output":"B"}
{"question_id":"hellaswag:00002767","item_index":328,"attempt":0,"prompt_hash":"920963d33d57808fbb8ada2c56998b71d224eacd550e1cc1c31ab8e7cf5df83b","question":"A female swimmer is standing on a diving board in a black swim suit and she jumps off doing multiple flips. she\n\nA. goes to the bottom of the pool and she does a flip up in the air, flips over and does a proper back flip.\nB. is then shown on ground level talking and wiping her face with a towel.\nC. waits for an instant and with incredible speed does a front flip on the diving board.\nD. then moves backwards and forward swirling around to bring her chest up to the diving board.","prompt":"A female swimmer is standing on a diving board in a black swim suit and she jumps off doing multiple flips. she","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6197,"error":null,"output":"B"}
{"question_id":"hellaswag:00006764","item_index":329,"attempt":0,"prompt_hash":"8e0e8385fd5dbac179bb193ca47429846db839fe511aecb717abf176bf5b3a42","question":"[header] How to apologize to a teen girl [title] Make a plan to meet up with her. [step] Apologies are best articulated when the parties involved are face to face. Apologizing in person means you set aside time out of your day specifically for this apology which will help her believe that you actually mean it.\n\nA. [substeps] Arrive at the bar just a few minutes early for the perfect meeting time. Her mood will be better with just a few minutes to spare.\nB. [substeps] You could ask her to please meet you briefly at the local coffee shop or library. Or try asking if it would be okay for you to stop by her house so she can be in a comfortable environment for your apology.\nC. [substeps] If you cannot get to see her in person, pick an appropriate time to talk to her. It might be a good time between 10 pm and 8 pm if there isn't too much down time in the forecast.\nD. [substeps] Don't text or meet up with her near the end of the night as it will be harder to study and she may become annoyed with you. You should make a phone plan to meet up for coffee about a half hour before class to catch up.","prompt":"[header] How to apologize to a teen girl [title] Make a plan to meet up with her. [step] Apologies are best articulated when the parties involved are face to face. Apologizing in person means you set aside time out of your day specifically for this apology which will help her believe that you actually mean it.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6193,"error":null,"output":"B"}
{"question_id":"hellaswag:00008272","item_index":330,"attempt":0,"prompt_hash":"2631d34a59d7412e8326679fcf02c47795cc761e9d801b605b8652d6e68e1030","question":"[header] How to store knee high boots [title] Zip up the sides of your boots in readiness for storing. [title] Roll up an unwanted magazine and insert into one boot. [step] ( you will need to hold the boot up as you insert the magazine.\n\nA. ) [title] Secure the magazine between the rear part of the boot and the sole of your foot using duct tape or dental floss, as if you were freezing. [title] Clamp down firmly onto the boot.\nB. ) [substeps] Keep the magazine between the back and bottom of both your left and right boots. This will help to protect everything on both boots.\nC. ) immediately it will uncurl and fill the boot shape. [title] Roll up a second unwanted magazine and insert it into the other boot.\nD. ) [title] Place the magazine down on a flat surface to reduce the chance of the magazine or boots getting rolled up. [title] Pick up a very light object such as a book or newspaper.","prompt":"[header] How to store knee high boots [title] Zip up the sides of your boots in readiness for storing. [title] Roll up an unwanted magazine and insert into one boot. [step] ( you will need to hold the boot up as you insert the magazine.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6193,"error":null,"output":"C"}
{"question_id":"hellaswag:00000501","item_index":331,"attempt":0,"prompt_hash":"69fef5fa8e3bd257037eced46d96dd0bf2bb698446deaa1813d64c41e3880cd9","question":"He puts a fishing pole into the ice. He pulls a fish out of the hole and puts it on top of the ice. he\n\nA. takes a fish out of the hole.\nB. picks up the fish in his hand.\nC. begins fishing from a hole in the ice.\nD. pushes in the center fish.","prompt":"He puts a fishing pole into the ice. He pulls a fish out of the hole and puts it on top of the ice. he","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6194,"error":null,"output":"B"}
{"question_id":"hellaswag:00002874","item_index":332,"attempt":0,"prompt_hash":"13197b543375afb87ca334c621b1f3d2b1075ce2891640066ec876f5b761f493","question":"First they cut the top off of a green pumpkin and scoop the seeds and other stuff out. The remaining insides are cut and scraped off with an ice cream scoop. a design\n\nA. is added to the pumpkin and they're boxed up and listed for display.\nB. is drawn over the inside of a pumpkin and an image and words are drawn along the insides.\nC. is then filled in on the pumpkin and the inside with icing and powdered sugar.\nD. is drawn on the pumpkin and then cut out.","prompt":"First they cut the top off of a green pumpkin and scoop the seeds and other stuff out. The remaining insides are cut and scraped off with an ice cream scoop. a design","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6197,"error":null,"output":"D"}
{"question_id":"hellaswag:00002625","item_index":333,"attempt":0,"prompt_hash":"83455b6a992e82f6f3d284fd9eb132f617e42f1bc7fa36d3f55b8684bab7595d","question":"We then see people lined up a the start of a a marathon take off running as words stream at the bottom of the screen. Asian script appears and a man stops running to look at the camera. the man\n\nA. throws money on the ground then taunts someone.\nB. then is shown running in a marathon while knees slightly bent from running on his waxed behind.\nC. runs off the marathon course and we then see shots of various location such as lakes, weddings, cities the ocean.\nD. throws the sword away and we see ending credits with the title at the bottom.","prompt":"We then see people lined up a the start of a a marathon take off running as words stream at the bottom of the screen. Asian script appears and a man stops running to look at the camera. the man","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6194,"error":null,"output":"D"}
{"question_id":"hellaswag:00001319","item_index":334,"attempt":0,"prompt_hash":"60c6a02f062689c22c785267decbc5d9dcc9fe948a613e9430cb70d36576e1c5","question":"Girls talk in a beach, then girls spray sunscreen on the back of women, also a woman wearing a white coat talks and shows products. a man\n\nA. sprays sunscreen on his back, and then he sprays sunscreen to the back of a woman.\nB. also talks holding a blue spray gun.\nC. washes a car then talks with the woman.\nD. hold a container on a bike and other women sprays the driver's side of a car.","prompt":"Girls talk in a beach, then girls spray sunscreen on the back of women, also a woman wearing a white coat talks and shows products. a man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6196,"error":null,"output":"A"}
{"question_id":"hellaswag:00002272","item_index":335,"attempt":0,"prompt_hash":"f59803820ad2c37feaca0c4d0e5a7639e6ddf1e8c8a32d30c68000229525a7af","question":"A pinata hangs from a tree, as a group of girls stand in front of it. One has a stick and is blindfolded. she\n\nA. throws the stick, and then jumps and spins before walking away.\nB. hits the pinata, then walks away.\nC. is picking up a small portion of the pinata.\nD. throws the stick, and peeks out for a few seconds.","prompt":"A pinata hangs from a tree, as a group of girls stand in front of it. One has a stick and is blindfolded. she","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6193,"error":null,"output":"B"}
{"question_id":"hellaswag:00001938","item_index":336,"attempt":0,"prompt_hash":"86b3665f5daf2931fc4a2a3a4cff3435d658d77c5e99e65c2d5c4402e68314ab","question":"A man stands in a driveway covered with snow talking and holding a shovel. Then, the man shovel the snow on the driveway and put in a pile. the man\n\nA. appears holding several lights and shovel the snow on the driveway.\nB. shows how to bend the legs, and then continues shoveling snow.\nC. moves a leaf off the driveway.\nD. drives a motorcycle while continuing to talk.","prompt":"A man stands in a driveway covered with snow talking and holding a shovel. Then, the man shovel the snow on the driveway and put in a pile. the man","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6196,"error":null,"output":"A"}
{"question_id":"hellaswag:00001751","item_index":337,"attempt":0,"prompt_hash":"d2bddd52303c495d2be0d9c991800aa16c2215cdc1493317a7bff0ba8a14615a","question":"A man is seen kneeling down before a set of weights while another watches on the side. he\n\nA. holds the weights up for leverage while the camera watches.\nB. lifts up the weights one at a time displaying how many they hit against the side.\nC. lifts up the weights and throws them back down.\nD. pulls off the straps and begins moving around while the camera zooms in on his movements.","prompt":"A man is seen kneeling down before a set of weights while another watches on the side. he","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6197,"error":null,"output":"C"}
{"question_id":"hellaswag:00005252","item_index":338,"attempt":0,"prompt_hash":"3b0f142614bc0cdb1e307c4a359532c295977aa8049cccb340fc2907f733ce64","question":"[header] How to speak supportively to someone with an eating disorder [title] Educate yourself. [step] Research what it means to suffer from an eating disorder, how people cope with it, and what steps can be taken to overcome the problem. It is important that you have at least a little bit of background knowledge so the person will listen to you and think that you are someone they can talk to about their problem.\n\nA. [title] Determine whether you are a social phobia. [step] Social phobia, or shame, can impact your perceptions of others and create resentment towards them.\nB. [title] Cultivate self awareness and helpful habits. [step] Observe the values and beliefs of others and ask yourself how they view the world.\nC. [substeps] Help them feel that they are telling you how much they need help. For example, ask them if they are feeling better or need help with something.\nD. [substeps] The more you learn about their condition, the better equipped you'll be to approach them with your concern. They may be suffering from extreme levels of anxiety or depression that you are unaware of.","prompt":"[header] How to speak supportively to someone with an eating disorder [title] Educate yourself. [step] Research what it means to suffer from an eating disorder, how people cope with it, and what steps can be taken to overcome the problem. It is important that you have at least a little bit of background knowledge so the person will listen to you and think that you are someone they can talk to about their problem.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6194,"error":null,"output":"D"}
{"question_id":"hellaswag:00002657","item_index":339,"attempt":0,"prompt_hash":"690f19fb273b4e89ebc0b74191d0902b52f01207cec06997c36c65f1e1bc7075","question":"A woman is sitting at a desk. A man sitting in front of her is playing a harmonica. the woman\n\nA. sits and decides to play with each other.\nB. starts talking to the guy.\nC. standing is looking to the camera.\nD. is playing with her hair.","prompt":"A woman is sitting at a desk. A man sitting in front of her is playing a harmonica. the woman","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6192,"error":null,"output":"D"}
{"question_id":"hellaswag:00001975","item_index":340,"attempt":0,"prompt_hash":"63280c5026cf7c42de34fd7e2b8edd2d5b72dd11a4a8a194fe471c157188d978","question":"A baby boy is seated at a restaurant table. he\n\nA. is eating a vanilla ice cream cone.\nB. is smoking a cigarette, blowing smoke rings.\nC. is using a rag to wipe a glass window.\nD. is smoking and laughing a lot.","prompt":"A baby boy is seated at a restaurant table. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6697,"error":null,"output":"A"}
{"question_id":"hellaswag:00000226","item_index":341,"attempt":0,"prompt_hash":"7b1d2762f2adc2fd753aad35429ae2e085a2772b927c38c316f986ac1be28cdc","question":"Man is standing on side of car stuck in traffic. man holding bagpipes\n\nA. is standing in front of a car and is playing them from the bottom of his shoe watching the people driving the cars around.\nB. is moving the car through traffic.\nC. is walking in street between cars.\nD. is talking to the camera anther different sidewalk in street.","prompt":"Man is standing on side of car stuck in traffic. man holding bagpipes","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6696,"error":null,"output":"C"}
{"question_id":"hellaswag:00009760","item_index":342,"attempt":0,"prompt_hash":"e2a5239116d2887e1a46b9b960fb7afbfc5a57ad5d05cdf031d8583078047bc4","question":"[header] How to choose a yoga studio [title] Decide on the style of yoga you'd like to practice. [step] Some studios offer a wide range of styles from gentle yin and restorative practices, to vigorous and challenging ashtanga and bikram. Other studios may focus solely on 1 style.\n\nA. [substeps] The main hallmark of yoga is that you have the freedom to choose your own style. This freedom is great for beginner and intermediate beginners.\nB. [title] Decide if you'd like to practice full lotus pose, thereby boosting your range of motion from head to feet. [step] This is particularly useful for beginners if you do a series of exercises on your feet.\nC. [substeps] If you're not sure what type of yoga you prefer, rent or borrow yoga videos to determine what you like, then look for studios in your area that offer that style. [title] Find a studio that fits your budget.\nD. [substeps] Nuana does not recognize some yoga styles. There are about 300 studios worldwide, so you want to know if you would like to be a good practitioner.","prompt":"[header] How to choose a yoga studio [title] Decide on the style of yoga you'd like to practice. [step] Some studios offer a wide range of styles from gentle yin and restorative practices, to vigorous and challenging ashtanga and bikram. Other studios may focus solely on 1 style.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6696,"error":null,"output":"C"}
{"question_id":"hellaswag:00006938","item_index":343,"attempt":0,"prompt_hash":"7b4b1c5e277f83ae85378cad4bf2403358dcb1662378b39095ceaf096c5fd075","question":"[header] How to care for a leopard gecko [title] Buy a 10-20 gallon (37.9-75.7 l) tank with a screen lid. [step] Purchase a glass or plastic container from a pet store to house your leopard gecko, including a securely fastened screen lid. A secure lid is important, especially if you have a cat.\n\nA. A lid will enable your leopard gecko to vocalise safely, and provide protection against harmful items. [substeps] Both reptiles and geckos need a lid that is wide enough that they can crawl and jump up and down in.\nB. [substeps] Leopard geckos should be kept in cages with which a single door opens. Bedding should be kept inside the tank, as leopard geckos do not need a lot of room to move and breathe.\nC. You may find these sold as aquariums, vivariums, or terrariums. If you already have a home for your gecko, skim through this section to make sure it meets your pet's requirements.\nD. Your gecko will not want to be disturbed during the day, and it will also need a bright light to get around in the dark. Large geckos may also require bigger screen screens.","prompt":"[header] How to care for a leopard gecko [title] Buy a 10-20 gallon (37.9-75.7 l) tank with a screen lid. [step] Purchase a glass or plastic container from a pet store to house your leopard gecko, including a securely fastened screen lid. A secure lid is important, especially if you have a cat.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6697,"error":null,"output":"C"}
{"question_id":"hellaswag:00004389","item_index":344,"attempt":0,"prompt_hash":"4dd2d091bf8e8d8fee6aeef1352a9a14f903af1478b5b682898726e54a491ac1","question":"[header] How to apply foundation and powder [title] Start with a clean face, then apply some toner and moisturizer. [step] Apply the toner first using a cotton ball, and the moisturizer using your fingers. The toner will help balance the ph of your skin.\n\nA. Aim for an oil-free finish, since you want to make sure your foundation is not too light or too heavy. [substeps] Moisturizer will also keep your foundation from smudging and breaking.\nB. It will also help tighten pores and make them less noticeable. The moisturizer will help keep your skin feeling smooth and soft.\nC. [substeps] Blend a tiny bit for a matte finish, but don't fill in large pores. A bit of mascara will work best too, but if you're going for classy glam streaks, you might want to start with a dark brown or black.\nD. If you have dry skin, apply some foundation first. [title] Take a piece of white paper or a cotton cloth and fold it in half.","prompt":"[header] How to apply foundation and powder [title] Start with a clean face, then apply some toner and moisturizer. [step] Apply the toner first using a cotton ball, and the moisturizer using your fingers. The toner will help balance the ph of your skin.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6696,"error":null,"output":"B"}
{"question_id":"hellaswag:00008983","item_index":345,"attempt":0,"prompt_hash":"b56ae3740d74d01d8c9d6a932f891de6b9e8e43c3844fa5c5549ae76634e3542","question":"[header] How to remove shellac nail polish [title] Apply cuticle oil to your cuticles. [step] Rub cuticle oil into the skin around your nails. Do not wipe away the excess.\n\nA. Simply wipe out the excess oil with a cuticle pusher or cotton swab. Leave the oil to rest for 10 to 15 minutes.\nB. Instead, wipe the oil off with a soft cloth. [substeps] This method is also known as priming.\nC. Use a cotton swab to remove the cuticle oil. [substeps] Try painting your cuticles lightly with the cuticle oil to avoid abrasions to the skin on your nails.\nD. [substeps] Cuticle oil is a product designed to soften and moisturize your cuticles, and it can be found at any major drugstore. Applying it to your cuticles before removing the shellac nail polish will provide your skin with a protective layer against the harsh, drying acetone needed for this process.","prompt":"[header] How to remove shellac nail polish [title] Apply cuticle oil to your cuticles. [step] Rub cuticle oil into the skin around your nails. Do not wipe away the excess.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6696,"error":null,"output":"D"}
{"question_id":"hellaswag:00006656","item_index":346,"attempt":0,"prompt_hash":"73839c93db06936f4d2432454d5407df77a2c0bd98efa0598f65a2e08dafde7b","question":"[header] How to buy sneakers [title] Seek out a sporting-goods or athletic shoe store. [step] Shop for exercise sneakers at an athletic specialty store rather than a department store. The selection of shoes will be wider, and store associates will be more knowledgeable about the types of shoes that suit different athletic pursuits.\n\nA. [substeps] Try walmart or target depending on the season you plan to compete in. You can also search the \" sporting goods \" directory for a shoe store near you.\nB. If you're going for a run, opt for one of these varieties of shoes while you can. Avoid sneakers with poorly-fit and scummy soles.\nC. [substeps] You may be able to find athletic shoes at most sporting goods stores. Working in teams at another store can help with this exercise.\nD. [substeps] Take your old shoes with you to the store. An associate can inspect them to gain valuable insights about how you exercise.","prompt":"[header] How to buy sneakers [title] Seek out a sporting-goods or athletic shoe store. [step] Shop for exercise sneakers at an athletic specialty store rather than a department store. The selection of shoes will be wider, and store associates will be more knowledgeable about the types of shoes that suit different athletic pursuits.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6695,"error":null,"output":"D"}
{"question_id":"hellaswag:00008124","item_index":347,"attempt":0,"prompt_hash":"3cb382d2e55ab8e8a667c6f83eec4353f858c16a873be1f469bfc530ed66a180","question":"[header] How to deal with death [title] Give yourself time to grieve. [step] The finality of the death may take a while to sink in, even if you were expecting the person to die. There is no \" normal \" timetable for grief; it is a personal journey.\n\nA. Let the emotions flow through you, and don't hold them in. [substeps] Many people feel that they shouldn't cry, get angry, or show any sort of emotion when someone dies.\nB. You have the choice whether or not to allow yourself this precious moment to pass without pain. Here are some general tips for easing your grief : [substeps] Don't go on with your life.\nC. A mourning person will have to be coping with loss and grieving in a peaceful and final way. Give yourself a few days to grieve, and use this time to begin a new activity and learn new techniques.\nD. The grieving process feels like a complex process through which you fill the empty spaces within your heart. Allow yourself some time to let your imagination run wild.","prompt":"[header] How to deal with death [title] Give yourself time to grieve. [step] The finality of the death may take a while to sink in, even if you were expecting the person to die. There is no \" normal \" timetable for grief; it is a personal journey.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6897,"error":null,"output":"A"}
{"question_id":"hellaswag:00008144","item_index":348,"attempt":0,"prompt_hash":"71752a06da9ab8c8b4ba25e60060f923196fb4d38945205eb65edf5421ca690c","question":"[header] How to clean a range hood [title] Choose the right cleaner for your specific hood. [step] Range hoods can be made from a variety of materials. When choosing what to clean with, pick a cleaner that's safe to use on that material in order to avoid damaging it.\n\nA. [substeps] Sweep and wipe down the hood and listen for a noise or vibration when you clean it. Alternatively, you could purchase a cleaning solution from your local hardware store.\nB. You can usually pick a specific cleaner, but you could also consider different cleansers, too. [substeps] For example, you could use a mild dish detergent based cleaner.\nC. [substeps] Classic vehicle hoods tend to have plastic walls and have holes where the hood has been torn or blown away. When choosing a cleaner, make sure the quality of the material you choose is appropriate for your user's needs and condition.\nD. For instance : [substeps] For plastic or vinyl hoods, use an all-purpose cleaner or warm soapy water. For stainless steel, go with warm soapy water.","prompt":"[header] How to clean a range hood [title] Choose the right cleaner for your specific hood. [step] Range hoods can be made from a variety of materials. When choosing what to clean with, pick a cleaner that's safe to use on that material in order to avoid damaging it.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6897,"error":null,"output":"D"}
{"question_id":"hellaswag:00009091","item_index":349,"attempt":0,"prompt_hash":"7ac8e74f7dfba24f293477a1182c3f266ab4aebf842e25f344ec5f8c51594341","question":"[header] How to towel dry hair [title] Get a soft towel or t-shirt ready. [step] When you're towel drying your hair, it's best to use a very soft towel or even an old t-shirt instead of a coarse, nubby towel. Using softer material will prevent your hair from getting tangled and damaged when you dry it.\n\nA. [substeps] Other general reasons you should towel dry your hair include night of sweat, need a shower, and going to the bathroom. You don't want to use a towel that is too rough.\nB. [substeps] Be careful not to bring the towel too close to your hair, or else it will bunch up under the towel. Towel drying dries your hair very quickly so be sure to only use soft towels as you dry your hair.\nC. [substeps] You could use any towel, but a clean, soft t-shirt could work for this step. [title] Use a hair dryer on low heat.\nD. [substeps] When you use a soft towel, your hair cuticle lies flat, so it dries in nice, glossy waves or curls. When you use a coarse towel, it's more likely to frizz.","prompt":"[header] How to towel dry hair [title] Get a soft towel or t-shirt ready. [step] When you're towel drying your hair, it's best to use a very soft towel or even an old t-shirt instead of a coarse, nubby towel. Using softer material will prevent your hair from getting tangled and damaged when you dry it.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6898,"error":null,"output":"B"}
{"question_id":"hellaswag:00008464","item_index":350,"attempt":0,"prompt_hash":"8ad6b58e3c447f1a2ab45d2dbe95247f0e49d12f12a5ded4cd7b54a1d73b62bc","question":"[header] How to make your long distance boyfriend feel special [title] Write an old-fashioned love letter to send by regular post. [step] While you can send an email instantly, getting a package, or letter in the post is a much more intimate and a heartfelt effort. Letters also create a keepsake that just can't happen via email.\n\nA. [substeps] Don't worry about writing the perfect poem or something profound, the fact that you made the effort to write is enough. You can draw a picture, find a heartfelt pre-made card or even send a postcard from a special place.\nB. [substeps] Consider creating a scrapbook of notes from your marriage as a way to surprise your boyfriend with flowers, a heartfelt necklace, or even a box of his favorite toys. The letter will go with most of his favorite movies, that you and your man enjoy talking about.\nC. [substeps] In a letter, you write words that extend deep into the person's heart. If you want to fool someone into telling you about a little secret you've told them, you have to be careful.\nD. [substeps] Write it silently, not loud and clear, and have the recipient read it twice. Definitely use the subject line to fill them in on why you're writing to your boyfriend.","prompt":"[header] How to make your long distance boyfriend feel special [title] Write an old-fashioned love letter to send by regular post. [step] While you can send an email instantly, getting a package, or letter in the post is a much more intimate and a heartfelt effort. Letters also create a keepsake that just can't happen via email.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6897,"error":null,"output":"A"}
{"question_id":"hellaswag:00001861","item_index":351,"attempt":0,"prompt_hash":"945a375d3eba8cd081ac1b4f84dc95f95f0a7d0f3f90c21a577c44b00b1af160","question":"The man went to the dirty water. the cupped the water\n\nA. bottle in his hand, then splashed and splash water on the man while he tighten and water splash the man.\nB. up to more seasonally.\nC. and wash his face.\nD. high to his head.","prompt":"The man went to the dirty water. the cupped the water","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6952,"error":null,"output":"C"}
{"question_id":"hellaswag:00003850","item_index":352,"attempt":0,"prompt_hash":"fb55eb9ab67f932a0aafcb52330b8052a1e4568873fe842bca2cb692ead20868","question":"[header] How to diagnose a herniated disc [title] Pay attention to pain in your lower back. [step] Most cases of herniated disc are in the lower back. You will notice a sharp or dull pain that may improve after a few days.\n\nA. At this point, it is called herniated disc. The pain should gradually subside after three to four days, and the sharp or dull pain may only return after three to four days.\nB. [substeps] You should also be aware of any discomfort you might feel in one particular area, usually of your lower back. The pain you feel when your upper back is in pain is typically described as a sharp, dull pain, with a delicate and soft feeling.\nC. Look for mild pain that does not indicate arthritis or stiffness along the spine. [title] Look for stuffiness around your upper back.\nD. [substeps] Your pain may go away in your back but move into your leg. You may not experience any pain with a herniated disc, but your doctor may have trouble diagnosing your condition if you don't.","prompt":"[header] How to diagnose a herniated disc [title] Pay attention to pain in your lower back. [step] Most cases of herniated disc are in the lower back. You will notice a sharp or dull pain that may improve after a few days.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4222,"error":null,"output":"A"}
{"question_id":"hellaswag:00003183","item_index":353,"attempt":0,"prompt_hash":"ced4684183a9e936884998efe8ccef384f5de26f432a641ad5325cde1723346b","question":"A person is chopping a mango on a red board. they\n\nA. mix different sized and colorful fruits together on a chopping block.\nB. stack the mango slices on a plate.\nC. put onion, vegetables, parsley, carrots and mayonnaise on a plate.\nD. pick up a red container as they chop the mango.","prompt":"A person is chopping a mango on a red board. they","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4226,"error":null,"output":"B"}
{"question_id":"hellaswag:00005344","item_index":354,"attempt":0,"prompt_hash":"98f1822a03e5f0689a54ba14ae0320771952cf41e09fca895cf1ca628afc4097","question":"[header] How to be happily married [title] Don't expect your happiness to be derived from someone else. [step] The most common mistake made in the quest for a happy marriage is to expect that someone should be \" making \" you happy. The old saying \" to have friends requires one to be a friend \" applies.\n\nA. Regardless of whether your happiness is due in some sense to someone you love or something to do with your own, you should hear and be happy. [substeps] Two people don't get married just because there is someone else there to provide love and companionship.\nB. Refrain from assuming that your friends and family love and care for you. In fact, expectations for your happiness are far from accurate.\nC. If someone is what you expect them to be, try to be a \" friend \" with them. They will likely be incredibly happy when they express their true love, and please, observe them for a moment before delving in deeper.\nD. [title] Focus on how you can make the marriage happy. [step] The only way you can be happy is by focusing on your own efforts to make your marriage partner happy.","prompt":"[header] How to be happily married [title] Don't expect your happiness to be derived from someone else. [step] The most common mistake made in the quest for a happy marriage is to expect that someone should be \" making \" you happy. The old saying \" to have friends requires one to be a friend \" applies.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4220,"error":null,"output":"D"}
{"question_id":"hellaswag:00002381","item_index":355,"attempt":0,"prompt_hash":"0f0e267a797bc960ddefec4f224d5d77803acc8cb57f9915fa6653fb96e33c6b","question":"Someone is holding a vacuum cleaner and turn it on to clean the floor someone is vacuuming the floor with an orange vacuum cleaner. different ingredients\n\nA. are used in various different parts of the vacuum and machine.\nB. are shown and the person then takes off the shoe and places it next to the trashcan the vacuuming is done with.\nC. are on the floor and someone cleans it.\nD. are added all at once.","prompt":"Someone is holding a vacuum cleaner and turn it on to clean the floor someone is vacuuming the floor with an orange vacuum cleaner. different ingredients","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4225,"error":null,"output":"C"}
{"question_id":"hellaswag:00006612","item_index":356,"attempt":0,"prompt_hash":"bacdee894a1edf97177e81c91d975e4184bc119eaeaf3dbbcfb01624b7719997","question":"[header] How to choose running shoes [title] Consider the surface you will be running on. [step] This will significantly influence what kind of shoe will be best for your foot. For instance, if you are running on a track or court specially designed to be flat and offer grip, you won't need extra traction on your shoe.\n\nA. Remember you'll need to balance your weight on your front foot and taking turns on your backwards leg. [substeps] Know that you won't have all the flexibility on a treadmill or tennis court.\nB. If you will be running on grass, for instance, you won't need much traction on your sole; just make sure you wear something that is especially stable and easily resistant. [title] Look for smooth, flexible soles.\nC. Generally : [substeps] Road/track running will require less traction and lighter weight. Trail running should have good traction, support, and cushion for uneven ground.\nD. [substeps] For example, you want a water-based shoe, or one which drains well from regular taps. You want to look for choices that are water-based; however, for instance, if the surface you intend to run on is sand, you need a different type of shoe.","prompt":"[header] How to choose running shoes [title] Consider the surface you will be running on. [step] This will significantly influence what kind of shoe will be best for your foot. For instance, if you are running on a track or court specially designed to be flat and offer grip, you won't need extra traction on your shoe.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4224,"error":null,"output":"B"}
{"question_id":"hellaswag:00000401","item_index":357,"attempt":0,"prompt_hash":"e0af5a5bb708116ce42ba8a7c59d1ccfd7b723678a17fe6505a76026ff2b7118","question":"A man is raking leaves in a yard. he\n\nA. raises it up to his face.\nB. stands up and goes back to raking.\nC. throws the leaves off into the distance.\nD. is using a clear bag to package them up.","prompt":"A man is raking leaves in a yard. he","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4223,"error":null,"output":"C"}
{"question_id":"hellaswag:00009820","item_index":358,"attempt":0,"prompt_hash":"6d9a3c0645a31dec89d9ddc5abb0eeda257bf01897b75adcf005e1157fdcfd57","question":"[header] How to organize craft supplies [title] Locate all of the craft supplies that you own. [substeps] If something is basically already organized or put away, don't pull it out now. Start with the stuff that's in your way.\n\nA. If it's still there, you're starting to waste time. When you start looking through each box from no clue there is no need for you to search.\nB. Putting things away is just a distraction and will reduce the amount of time you need to worry about it. Self-designated storage spaces can make this home more relevant to crafting ideas.\nC. Don't be afraid to be a little willing to put your art gear away and put the out-of-place items away. For instance, for instance, a few of your favorite clothes needs to be put away, and you want to put them away for free.\nD. Go a bit at a time, especially if you have a lot of stuff. Spend fifteen minutes sorting, or just sort one bag, bin, or area.","prompt":"[header] How to organize craft supplies [title] Locate all of the craft supplies that you own. [substeps] If something is basically already organized or put away, don't pull it out now. Start with the stuff that's in your way.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4218,"error":null,"output":"D"}
{"question_id":"hellaswag:00009539","item_index":359,"attempt":0,"prompt_hash":"d685b05406af9edbb7fd6e55f8e0d0f9d712326b90c6a533a84258774a7f900f","question":"[header] How to boost your business image [title] Have an \" elevator pitch \" ready to use. [step] If you meet someone in an elevator or on line at a coffee shop, be prepared to say something about your company that gets your message out briefly. For example, if you make designer jewelry, you might say, \" you have pretty earrings.\n\nA. \" [substeps] Having an elevator pitch is a chance to demonstrate to the recipient that you know a little bit about business. For example, you could say, \" i found out recently that your sales are low and that you're looking for the perfect marketing location to utilize your jewelry line.\nB. What makes you look like that? \" [title] Be deprecating. [step] Avoid irony and start with some humor at the beginning so you build a foundation for your message.\nC. I just started a company that makes designer jewelry you might like. \" several sources are available online for more information about developing a strong elevator pitch.\nD. I would love to take them with me to the show. \" you might say something like, \" i'm a musician and enjoy learning how to play the guitar.","prompt":"[header] How to boost your business image [title] Have an \" elevator pitch \" ready to use. [step] If you meet someone in an elevator or on line at a coffee shop, be prepared to say something about your company that gets your message out briefly. For example, if you make designer jewelry, you might say, \" you have pretty earrings.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4219,"error":null,"output":"C"}
{"question_id":"hellaswag:00000523","item_index":360,"attempt":0,"prompt_hash":"8928bda8896caf67b8834d798b0c36619e922292f06c0a9ff1b01c7e1a0919cf","question":"A large truck cake is shown, a smaller one next to it, and the toy truck is also there. the man\n\nA. uses a hand squeegee to train the toy truck, and holds it extended for inspection until the toys are empty.\nB. cleans the christmas tree, he picks up supplies, and makes sure the machine is running.\nC. is now pouring more lotion onto his front in piles.\nD. is back at the large light gray counter and in a sped up motion the man is shown with various different supplies as he assembles a very large truck cake that was previously shown.","prompt":"A large truck cake is shown, a smaller one next to it, and the toy truck is also there. the man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4222,"error":null,"output":"D"}
{"question_id":"hellaswag:00003462","item_index":361,"attempt":0,"prompt_hash":"0fec4327061a11b299360c528e8b7568108e9f77b50acdd0f9bb5ffccbc546f1","question":"[header] How to deal with stress when watching kids [title] Practice deep breathing. [step] Stress causes the \" fight or flight \" response in your body to kick in. Your heart beats faster, you may have trouble focusing, and your breath becomes shallow.\n\nA. Deep breathing is a great exercise to practice when you're feeling stressed and overwhelmed. Controlling your breath sends the message to your body that it's okay to relax.\nB. To combat stress while watching kids, inhale, hold for a second, and then exhale. Close your eyes and repeat the pledge nine times in your head.\nC. If this happens, try breathing in and out slowly from your diaphragm. Practice deep breathing for about 10-15 minutes each day.\nD. When you're looking for a calming stimulus, deep breathing can be the key. [substeps] Sit in a quiet space such as a room or kitchen and breathe in and out deeply.","prompt":"[header] How to deal with stress when watching kids [title] Practice deep breathing. [step] Stress causes the \" fight or flight \" response in your body to kick in. Your heart beats faster, you may have trouble focusing, and your breath becomes shallow.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4219,"error":null,"output":"A"}
{"question_id":"hellaswag:00005023","item_index":362,"attempt":0,"prompt_hash":"d7ebee94645277676eecb72cab3afaa4368df13a7ccde34aca8c6770091b2fe6","question":"[header] How to put a friend or relative out of your house [title] Determine why you want them to leave. [step] You need to be clear with your own reasoning before diving into the conversation. Review any agreements you made when they moved in, or any promises that have been made/broken.\n\nA. Ask yourself why you want them to stay. [substeps] Resolving the issue isn't easy.\nB. Assess the situation and their current behavior, grounding your reasoning in fact. While \" i don't like living with them \" is an acceptable reason to ask someone to move, you want concrete details, like \" they never do the dishes, \" \" they said they would leave months ago, \" etc.\nC. [substeps] Think carefully about whether or not you want them to stop staying at your house. You don't always have to grant your friend or relative permission.\nD. [substeps] Think about asking a favor or requesting that someone stay. Nothing is more amusing than sitting down with them (and maintaining eye contact) and asking them a favor that is completely unsolicited without talking to them first.","prompt":"[header] How to put a friend or relative out of your house [title] Determine why you want them to leave. [step] You need to be clear with your own reasoning before diving into the conversation. Review any agreements you made when they moved in, or any promises that have been made/broken.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4218,"error":null,"output":"C"}
{"question_id":"hellaswag:00006477","item_index":363,"attempt":0,"prompt_hash":"15d995c2acc864fb5dd56b77a82cb8951a1ae883dfff7864c9fc04e88a50a46d","question":"[header] How to adopt a cat through a rescue organization [title] Locate a rescue organization near you. [step] There are several ways to locate a rescue organization in your area. If you have a cat or another pet already, you could ask your vet for a recommendation.\n\nA. If you know other rescue organizations, you may have them arrange for a trip to the nearest shelter or rescue center to help you find the right one. [substeps] Look up a rescue organization in your area online, calling their hotline at 1-800-273-talk.\nB. You might also ask friends or family members who have successfully adopted a cat via a rescue organization. Finally, you could search for a rescue organization online by typing a word string like \" rescue organization [your town] \" into your preferred search engine.\nC. If you have another cat or other pet, ask your vet about places where the rescue organization has been hiding their food and water in the last few years. [title] Ask the shelter or aspca if they would recommend cat food.\nD. If you don't have a rescue organization, search online or check with local pet organizations. [substeps] Petfinder is an online search tool that will ask if there are local animal shelters near you.","prompt":"[header] How to adopt a cat through a rescue organization [title] Locate a rescue organization near you. [step] There are several ways to locate a rescue organization in your area. If you have a cat or another pet already, you could ask your vet for a recommendation.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4222,"error":null,"output":"B"}
{"question_id":"hellaswag:00008330","item_index":364,"attempt":0,"prompt_hash":"c53dcb534e0da68fa786eb305628c6a489ee0e01edbdd0c4e026e7ae8cfe8371","question":"[header] How to ask someone to be your ring bearer [title] Pick a child that is three to seven years old. [step] A general rule for the ring bearer is a child between three and seven years old. This is a wide range of development, so take it as a guide.\n\nA. [substeps] Use any age, interests, and accomplishments you have in mind when asking her. Ideally, choose a child of the opposite sex.\nB. If your child is very young and development is slow, the earlier this child becomes, the more quickly he/she can start growing, and the more self-sufficient he/she will be. [substeps] Parents are okay with two or three young toddlers.\nC. [substeps] You can say something like \" my old grandmother took me to see a jeweler early in my life. I looked back and saw that she used diamond jewelry.\nD. This ensures that the child can walk and that they'll be able to perform the duties you need them to. [substeps] If you go with a child under three, you'll increase the chances of them crying, being distracted, falling, or generally not quite being able to get from point a to point b.","prompt":"[header] How to ask someone to be your ring bearer [title] Pick a child that is three to seven years old. [step] A general rule for the ring bearer is a child between three and seven years old. This is a wide range of development, so take it as a guide.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4219,"error":null,"output":"D"}
{"question_id":"hellaswag:00002237","item_index":365,"attempt":0,"prompt_hash":"4327535eec7b04d363b99c2f51e3a05394ef1968af872e8870cb15b3571da02e","question":"The man stretches and puts on his winter clothes. He sprays his shovel with nonstick spray and shovels his driveway. the man\n\nA. takes a piece of paper off of his windshield and writes on it.\nB. sits in his perfectly cut snow shovel and shovels while showing off the home he has helped create.\nC. lays down and had a hard time breathing.\nD. brags of his project.","prompt":"The man stretches and puts on his winter clothes. He sprays his shovel with nonstick spray and shovels his driveway. the man","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4220,"error":null,"output":"A"}
{"question_id":"hellaswag:00006335","item_index":366,"attempt":0,"prompt_hash":"d011c9e74317364296568460312a3f6bce18fffd388aa9f662f51692b1205b0f","question":"[header] How to organize toys [title] Designate a storage space for each type of toy. [step] If toys are taking over your home, split up storage space by toy type. Split up your child's bedroom, or your living room, into different sections.\n\nA. Put bins or other storage devices on desks and under your beds, allowing kids to grab them for puzzles, coloring, and other activity-related activities. [substeps] You don't need a living space for toys if you live nearby, but a storage space for everyday toys can improve the overall sense of organization.\nB. Each section can house a different type of toy. [substeps] In a child's bedroom, separate corners of each room for different toy types.\nC. This will make it easier for you to work together and do everything. [substeps] You can organize toys by age, genre, and time.\nD. This will create a more spacious space for each toy. [substeps] Designate a box or basket for each toy.","prompt":"[header] How to organize toys [title] Designate a storage space for each type of toy. [step] If toys are taking over your home, split up storage space by toy type. Split up your child's bedroom, or your living room, into different sections.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4217,"error":null,"output":"B"}
{"question_id":"hellaswag:00001681","item_index":367,"attempt":0,"prompt_hash":"565b8d697ab840df96032721336288b32f2cff63b0821f4b00b523f497d18fa3","question":"A woman is seated in a beauty chair. a tattooed man with tools\n\nA. is shaving some young girl's legs.\nB. and a tattoo artist are injecting her into a purple bottle.\nC. appears and pierces her lower lip.\nD. is walking across her body and she begins to laugh.","prompt":"A woman is seated in a beauty chair. a tattooed man with tools","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4218,"error":null,"output":"C"}
{"question_id":"hellaswag:00004444","item_index":368,"attempt":0,"prompt_hash":"661184d89873d1895e4b94bd2b313c8b52dbf8594d66aff46096766e6a9bc74d","question":"[header] How to snapchat a girl [title] Add her on snapchat. [step] Open snapchat and tap the ghost icon at the screen's top. In the resulting menu, select \" add friends.\n\nA. \" tap the \" create new group \" button (start logging in one of the following ways). This will create a new group for you.\nB. \" below that is a list of friends who have previously added the person or you. Scroll down the list to tap a friend.\nC. \" after that, you'll be able to add her with her username, through your phone's contacts (though you'll need to already have her number), or with a special snapchat code. [substeps] If you don't have her username, number, or snapchat code, you can always add her by finding her in the friends' list of a mutual friend.\nD. \" a small window will pop up telling you if your selected people have successfully added friends. You can add plus one of her friends on facebook, twitter, or other other social networking platforms.","prompt":"[header] How to snapchat a girl [title] Add her on snapchat. [step] Open snapchat and tap the ghost icon at the screen's top. In the resulting menu, select \" add friends.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4218,"error":null,"output":"C"}
{"question_id":"hellaswag:00004642","item_index":369,"attempt":0,"prompt_hash":"af833fcce492f0b22c9be8a9b5b83fc86ed7449ed53b31525ad3d2cfe0650668","question":"[header] How to dress like an alpha [title] Hair can make or break your entire outfit. [step] So make sure it looks good. No frizzy hair, no out of place hair, and definitely no greasy hair.\n\nA. Buy moroccan oil to control frizz, make hair soft, and add tons of shine. [substeps] Shampoo/condition everyday (or every other day, at the very least).\nB. [title] Take some time to shower before you leave the house. [step] This is supposed to ensure that your hair looks great and that nothing is wrinkled.\nC. Don't wear out classy wear the same thing all year! So when cold and winter, wear classy do your best to pull it off. Keep messy, naturally wavy, and girly and your hair will look amazing and you will look awesome.\nD. Make sure that it only adds body to your look. Wear it down or put it up a little.","prompt":"[header] How to dress like an alpha [title] Hair can make or break your entire outfit. [step] So make sure it looks good. No frizzy hair, no out of place hair, and definitely no greasy hair.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4217,"error":null,"output":"B"}
{"question_id":"hellaswag:00009047","item_index":370,"attempt":0,"prompt_hash":"d285edcccaa6cf8d3a6f61cbf3af61888e8d988107f9f0dacdd3aa1021c28384","question":"[header] How to read and understand romeo and juliet [title] Go to your local bookstore and purchase a book that tells the story in simpler words. [step] Normally these books are called \" shakespeare made easy \", and cost around $6.99. [title] Look at the two sides of the book.\n\nA. [step] The page is located between the two sides. It usually has one letter marked \" d \", with an arrow pointing to the left of each letter.\nB. [step] Should they change colors-be sure the one with a subtitle (ex. William shakespeare) gets the wrong colors! Don't read the first half of the book.\nC. [title] Look at each of the introductory words. [step] Name one of the two to determine what it means, and whether the second side means something at all to him.\nD. [step] You'll notice that one side is written in shakespearean language and the other side is english translated. Read the shakespearean and when you come along a difficult spot, look to the other side for help.","prompt":"[header] How to read and understand romeo and juliet [title] Go to your local bookstore and purchase a book that tells the story in simpler words. [step] Normally these books are called \" shakespeare made easy \", and cost around $6.99. [title] Look at the two sides of the book.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4218,"error":null,"output":"D"}