{"question_id":"hellaswag:00001856","item_index":0,"attempt":0,"prompt_hash":"7dc0758352f81f06997cfce215e867cbe6b9cf46da965759ab455cc491abf5de","question":"A group of men stand on top of logs, chopping away. The chopping legend speaks to the camera again. a couple of men\n\nA. are in a group near a river, talking.\nB. stand on the log top wearing protective gear.\nC. chop some wood until one of them jumps off the log and raises his hands up in excitement.\nD. are hanging up rocks above a river.","prompt":"A group of men stand on top of logs, chopping away. The chopping legend speaks to the camera again. a couple of men","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4681,"error":null,"output":"C"}
{"question_id":"hellaswag:00008393","item_index":1,"attempt":0,"prompt_hash":"95202ba8c5597444fb7e99430ef7b871427d7f5409a75b607edf8b521b51244d","question":"[header] How to use movies to help kids build character [title] Pick movies by age group. [step] While some younger kids are more mature, generally, you don't want a story line that will go right over their head. Similarly, you don't want to lose older kids by showing them movies that are too simplistic.\n\nA. [substeps] Often, the best movie lines are set in the last century. While you may have super teens, you can find lots of great movies for kids online.\nB. If possible, choose movies by kids you love, not kids they already like. [substeps] For instance, in the movie audrey hepburn 2 2000, the tables at breakfast [[ (18.\nC. Try to pick something that works for the age the kid is at. [substeps] For kids 2 to 7 years old, make sure the story is fairly simple, and that plot supports the idea you're trying to get across, as it needs to be glaringly obvious.\nD. Most of the time, kids get what they want early on. Take your age group into consideration as you come up with a few choices for your movie series.","prompt":"[header] How to use movies to help kids build character [title] Pick movies by age group. [step] While some younger kids are more mature, generally, you don't want a story line that will go right over their head. Similarly, you don't want to lose older kids by showing them movies that are too simplistic.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4679,"error":null,"output":"D"}
{"question_id":"hellaswag:00007942","item_index":2,"attempt":0,"prompt_hash":"a4d6d883f5bb80f1250d8e76fa8927e29aed5fc9dcd8629215e24695dec51642","question":"[header] How to choose an alternative to a flower girl [title] Invite other members of the family to join your wedding party. [step] Instead of having young flower girls, consider using older women in your family, such as your grandma or great aunt. They'll love to be included in a special way, and the sight of your grandma throwing flower petals into the air is to sure to bring a smile to everyone's face.\n\nA. [substeps] Older women can even invite their significant other as well. Just be sure the couple makes time to discuss the details in advance.\nB. [title] Let everyone know you're having a flower girl themed party. [step] In several ways, this will make your party more fun.\nC. [title] Read opinions from other members of the family. [step] Reading magazines and articles on the flower girl instead of relying on your friends for opinions can help you understand what happened that night in the picture.\nD. [title] Give the role to a friend who volunteers. [step] One of your adult friends may find the idea of being a flower girl enchanting.","prompt":"[header] How to choose an alternative to a flower girl [title] Invite other members of the family to join your wedding party. [step] Instead of having young flower girls, consider using older women in your family, such as your grandma or great aunt. They'll love to be included in a special way, and the sight of your grandma throwing flower petals into the air is to sure to bring a smile to everyone's face.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4677,"error":null,"output":"D"}
{"question_id":"hellaswag:00007936","item_index":3,"attempt":0,"prompt_hash":"661a83c3c67a2d018d44552cb59fd82f705be2b60cd757730c2647b9c4ba4e8d","question":"[header] How to care for your skin before and after a brazilian wax [title] Prepare 24 hours before wax. [step] The day before your wax, exfoliate the area. You don't need to use a special scrub.\n\nA. You can exfoliate regularly with a loofah, facial bar, or regular washcloth. [substeps] Do not exfoliate right before you wax your brows.\nB. All you need is a washcloth or bath pouffe with soap. Make sure to do this at least a day before your wax.\nC. [substeps] As you rub on and dry your skin afterwards, avoid washing with soap or body wash. [title] Exfoliate with a facial cleanser.\nD. [substeps] To get the most benefit, scrub with a pumice stone. If you are not exfoliating, begin by exfoliating with a pumice stone the day before wax.","prompt":"[header] How to care for your skin before and after a brazilian wax [title] Prepare 24 hours before wax. [step] The day before your wax, exfoliate the area. You don't need to use a special scrub.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4676,"error":null,"output":"D"}
{"question_id":"hellaswag:00005966","item_index":4,"attempt":0,"prompt_hash":"5bb3a5ea3513377fc7f0bfe1b4e5e058176d748f67ae20f1ad1b3e9b79b7990a","question":"[header] How to make resurrection rolls [title] Pour the milk into a bowl. [step] To activate the yeast, you need to mix it with a warm liquid. Add ½ cup (118 ml) of warm milk to the bowl of a stand mixer.\n\nA. [substeps] For a longer process, you might want to run the yeast in the sink before proceeding. [title] Gradually stir the milk and mixture with a whisk.\nB. Mix until the milk is very light (around 110 ml). [substeps] If the milk is too soft for this recipe, add 1/2 cup (120 ml) of sour cream.\nC. The milk should be 105 ° f (41 ° c). [substeps] You can use 1% or 2% milk, but whole milk usually results in the best rolls.\nD. [substeps] If you have a hand mixer, you can make the dough for the roll yourself. You'll only need to make 2 cups (500 ml) of milk.","prompt":"[header] How to make resurrection rolls [title] Pour the milk into a bowl. [step] To activate the yeast, you need to mix it with a warm liquid. Add ½ cup (118 ml) of warm milk to the bowl of a stand mixer.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4679,"error":null,"output":"C"}
{"question_id":"hellaswag:00003322","item_index":5,"attempt":0,"prompt_hash":"c48fcbbe29dae6d2bff97d9c731d682f11441dab1137d906fd00e05951428a67","question":"[header] How to wear a sports bra [title] Select a sports bra made of moisture-wicking material. [step] You want your sports bra to be made of moisture-wicking material that is breathable. Most new sports bras these days contain technology to wick away sweat, making them the ideal choice for working out.\n\nA. [title] Choose a sports bra made of breathable materials. [step] High-quality sports bras should not be too loose-fitting because this could send heat into the necessary space and encourage sweat from the hard to reach tissues in your body.\nB. This makes them perfect for work or after sports practice. [substeps] In addition to sweat, sports bras also help to protect your arm and leg from heat, and can go well under shorts or other form-fitting gear.\nC. [substeps] Not everyone has how much of the material they need for a workout in their bras. Less is more, as sport bras tend to last longer.\nD. Try to steer clear of cotton, which tends to soak up moisture and stay wet. [substeps] Choosing a moisture-wicking material will also help to regulate the temperature of your body as you're working out.","prompt":"[header] How to wear a sports bra [title] Select a sports bra made of moisture-wicking material. [step] You want your sports bra to be made of moisture-wicking material that is breathable. Most new sports bras these days contain technology to wick away sweat, making them the ideal choice for working out.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4681,"error":null,"output":"D"}
{"question_id":"hellaswag:00002823","item_index":6,"attempt":0,"prompt_hash":"18b765afcb412d9149ae8d35218bafee36f2b54cc248d8e8555c03a8edee182f","question":"A black screen appears with red and white letters on it that say \"discus throw example of correction of a delivery problem for a beginner\". a young lady\n\nA. takes her fist into the air and jumps to the right, fast, catching several discs in the air.\nB. walks into a white room wearing a black jacket and reappears with an aluminum fork in her hand.\nC. appears for the discus throw classroom at some point in between.\nD. is now standing on a concrete circle and she begins swinging her red discus, then swings her body and then throws it.","prompt":"A black screen appears with red and white letters on it that say \"discus throw example of correction of a delivery problem for a beginner\". a young lady","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":8414,"error":null,"output":"D"}
{"question_id":"hellaswag:00008354","item_index":7,"attempt":0,"prompt_hash":"7baba6e236e53df26210b84e0e5a53e8cbc51de98a4c3d6bf08772c107626982","question":"[header] How to eat pumpkin seeds [title] Preheat oven to 375 ° f (190 ° c). [title] Separate any pumpkin seeds from the pumpkin flesh. [step] The best way to do this is by hand, or perhaps by using an old (cleaned) comb that you wouldn't mind getting dirty.\n\nA. [title] Spread a sticky oil over this pumpkin and spread this onto your hands. [step] Allow to absorb, or sit for longer if you'd like.\nB. Use the comb to separate the pumpkin seeds from the fibrous, fleshy parts. [title] If you choose to, clean the outer shell of the pumpkin seeds.\nC. [title] Pour 1 teaspoon of pumpkin flesh into a glass. [step] Alternatively, you can use a spoon to gently separate the pumpkin seeds from the pumpkin flesh.\nD. Rinse all pumpkin seeds thoroughly in a warm and soapy sink and pat dry with paper towels. [title] Scoop the pumpkin seeds into the mini muffin muffin pans or mini paper baskets.","prompt":"[header] How to eat pumpkin seeds [title] Preheat oven to 375 ° f (190 ° c). [title] Separate any pumpkin seeds from the pumpkin flesh. [step] The best way to do this is by hand, or perhaps by using an old (cleaned) comb that you wouldn't mind getting dirty.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4677,"error":null,"output":"C"}
{"question_id":"hellaswag:00008071","item_index":8,"attempt":0,"prompt_hash":"bb985af4e8b4e5f9f06e48bbc4067cefbfba130f90f47daad6d6f23bb2d05a78","question":"[header] How to deal with your child being expelled from school [title] Remain calm. [step] Whether you've just gotten the news or are about to attend a meeting, remaining calm will help you problem-solve and approach the situation with a clear head. You might instantly take your child's side or feel like their expulsion is a reflection of you.\n\nA. But yelling, expressing hurt or anger, and screaming will only make matters worse by escalating the situation. [substeps] If you're going to talk with your child, give yourself permission to cry, to scratch, or to calm down.\nB. If you get a phone call or have a meeting with the school, do your best to be calm throughout. [substeps] If you notice you're feeling upset and need to calm down, take some deep breaths.\nC. [substeps] If you feel angry or upset, take two deep breaths. Both the release of anger and tension and the knowledge that your child is being expelled from a school will make you feel much more powerful.\nD. When you're really frustrated with the situation, try a reasoned response-\" mom, i just got today's news and it is very upsetting. That's why i am upset, \" or \" i am sorry our student is expelled.","prompt":"[header] How to deal with your child being expelled from school [title] Remain calm. [step] Whether you've just gotten the news or are about to attend a meeting, remaining calm will help you problem-solve and approach the situation with a clear head. You might instantly take your child's side or feel like their expulsion is a reflection of you.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":8414,"error":null,"output":"B"}
{"question_id":"hellaswag:00006530","item_index":9,"attempt":0,"prompt_hash":"0486300da8a0bad62c83a7ac64ebf42f7c6e60c93693dcd22e71647d73ebbcf9","question":"[header] How to prepare for a photographic trip [title] Bring a camera or cameras. [step] If at all possible, it is a good idea to have a backup. If you have some type of mechanical or power failure, you will not be able to take any pictures without a backup.\n\nA. You will be kneeling in front of a 75 watt bulb and your camera and a tripod might end up distracting you from viewing the landscape. [substeps] If you are planning to take a portrait, you will want to have a photograph of yourself and someone else and a camera that allows you to take screenshots or arrows.\nB. [title] Be sure that you have a flash or a strobe. [step] Your camera may have that and if you are used to using that on camera flash, then use it.\nC. A plastic camera can often deal with this problem. If you cannot bring a camera, you may want to ask a family member or friend to hold a spare camera for future use.\nD. So make sure you have a backup ready. [substeps] You can either purchase or borrow a brush to use during your trip.","prompt":"[header] How to prepare for a photographic trip [title] Bring a camera or cameras. [step] If at all possible, it is a good idea to have a backup. If you have some type of mechanical or power failure, you will not be able to take any pictures without a backup.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4678,"error":null,"output":"C"}
{"question_id":"hellaswag:00004667","item_index":10,"attempt":0,"prompt_hash":"d51828fe39573fb23d8dc56e80525b1fba0839f034c6fa396e56004bbd0dcdfd","question":"[header] How to do stiletto nails [title] Gather your materials. [step] Stiletto nails are easy to achieve, but you do need some special materials and tools. You will need : [substeps] A package of long, square shaped fake nails.\n\nA. Tacky nail polish remover to remove loose polish from the tips of fake nails. A temporary lash station or a separate eyelash curler.\nB. Get the kind that go over your entire nail. You can find these in beauty supply stores.\nC. Two pieces of cardboard or rectangular paper a ruler three different colored jewels, thumbtacks, or duct tape some glue nail polish a nail sealer or nail salon. [title] Cut the cardboard or paper to the correct length.\nD. Scissors for making very even cuts, or, you can use a craft knife. Nail clippers or the small cylindrical object that can be used as a nail clipper.","prompt":"[header] How to do stiletto nails [title] Gather your materials. [step] Stiletto nails are easy to achieve, but you do need some special materials and tools. You will need : [substeps] A package of long, square shaped fake nails.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4679,"error":null,"output":"B"}
{"question_id":"hellaswag:00007902","item_index":11,"attempt":0,"prompt_hash":"ec3dbbcd9539484554566d9b40bbb4d63523d7bad12ca760ab9aec4597b25680","question":"[header] How to help someone with depression and anxiety [title] Learn about depression and anxiety. [step] Educate yourself about both depression and anxiety. As the saying goes, \" knowledge is power, \" and this is true in helping someone with mental illness.\n\nA. The first step is to have a specific conversation with your friend or relative. Explain to your friend how you feel and what you want to do about the problem and how you want your friend to cope with it.\nB. Not only do you have to know about both, but you have to know how they feel about depression. This knowledge will help your relationship with someone change, including improving their ability to interact socially, and help them overcome the fear and anxiety that's responsible for depression.\nC. Using the above research, you can help someone you care about deal with mental issues whenever they arise, so it's important that you know when they may need you. [substeps] In addition to learning about depression, you can attend support groups for people who have depression or anxiety.\nD. The more you understand anxiety and depression, the better you can respond to someone's needs. Do some research about anxiety and depression online and/or talk to a psychologist, psychiatrist, therapist, or medical doctor.","prompt":"[header] How to help someone with depression and anxiety [title] Learn about depression and anxiety. [step] Educate yourself about both depression and anxiety. As the saying goes, \" knowledge is power, \" and this is true in helping someone with mental illness.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4933,"error":null,"output":"D"}
{"question_id":"hellaswag:00000696","item_index":12,"attempt":0,"prompt_hash":"8b9dc918e537e6f42c00dfce0e749a70fc154cf3a4197fd5013b87927f80f824","question":"A black screen appears and the white words the mikan drill are displayed and the man continues to talk and then demonstrates exactly how to do various mikan drills. the man\n\nA. then dumps out their clean clothing and they are then seen washing their clothes in a dryer and then placing the dirty clothes on stage.\nB. then grabs a second ball and continues to demonstrate advanced mikan drills with both balls.\nC. continues talking, exercises, and demonstrating how to properly properly do his all distance but 11 hour stay wakeboarding on a playground that is attached to a mobile tree.\nD. continues to talk and then walks back and fourth and demonstrates more mikan drills that involve using an electric drill to drill through a coconut in a board with a handle.","prompt":"A black screen appears and the white words the mikan drill are displayed and the man continues to talk and then demonstrates exactly how to do various mikan drills. the man","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4681,"error":null,"output":"B"}
{"question_id":"hellaswag:00006460","item_index":13,"attempt":0,"prompt_hash":"d133d935da49129a4723c6b537af16c025ab9d4f8fc1802942772a57fefb4ccb","question":"[header] How to make feta salsa [title] Chop the tomatoes, olives, and herbs. [step] Take 2/3 cup (35 g) of sundried tomatoes packed in oil and remove them from the jar. You'll also need 1/2 cup (67 g) of pitted kalamata olives.\n\nA. Put them on the counter and leave them there for a couple of minutes. Put some parsley on the olives as well.\nB. Place these on the pan with the tomatoes. [substeps] If you don't have agosi olives, you could use tapioca ripato.\nC. Use a sharp knife to carefully chop the tomatoes and olives into rough pieces. Place these in a serving bowl along with : [substeps] 2 tablespoons chopped fresh dill 3 tablespoons chopped flat-leaf parsley 2 thinly sliced scallions [title] Crumble the feta.\nD. Add 1/2 cup (59 ml) of all-purpose flour to the tomatoes. Mix well to coat all of the ingredients.","prompt":"[header] How to make feta salsa [title] Chop the tomatoes, olives, and herbs. [step] Take 2/3 cup (35 g) of sundried tomatoes packed in oil and remove them from the jar. You'll also need 1/2 cup (67 g) of pitted kalamata olives.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4679,"error":null,"output":"C"}
{"question_id":"hellaswag:00006432","item_index":14,"attempt":0,"prompt_hash":"3a1949a55cfa3ec1a091746d354adccd92654841289086fa67b8ac5e556d393d","question":"[header] How to make vegan mint chocolate cake [title] Start preheating the oven to 325 degrees f (160 c) as you coat your cooking pan with oil or cooking spray. [title] Place the beets in a saucepan and fill it with enough water to cover the beets. [title] Allow the water to come to a boil and cook them at a medium-high heat until they become soft when pierced with a fork.\n\nA. [title] Add the fresh mint and let the beets simmer, stirring frequently, for a few minutes. [title] Cover with plastic wrap and allow the beets to simmer for up to five minutes.\nB. [title] Remove the beets from the pan and place in a bowl. [step] The sieve will remove any dirt, sweat or bacteria clinging to the beets.\nC. [title] Turn off the heat and place the beets in a large bowl. [step] Stir in the melted butter, chocolate syrup, and the vanilla extract.\nD. [title] Allow the beets to cool off the heat as you place a fine mesh sieve over a medium bowl. [title] Pour the water and beats into the sieve.","prompt":"[header] How to make vegan mint chocolate cake [title] Start preheating the oven to 325 degrees f (160 c) as you coat your cooking pan with oil or cooking spray. [title] Place the beets in a saucepan and fill it with enough water to cover the beets. [title] Allow the water to come to a boil and cook them at a medium-high heat until they become soft when pierced with a fork.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4681,"error":null,"output":"D"}
{"question_id":"hellaswag:00006170","item_index":15,"attempt":0,"prompt_hash":"dbfc2c4a2c8ecc40bdf17916b346c70d688fc404396a01f50c27ff9c873ce1cf","question":"[header] How to create star nail art [title] Gather your materials. [step] Painting simple stars on your nails is much easier than you might think. You just need the right tools to do it.\n\nA. Wood work, scissors, a nail cuticle pusher, a stain remover, and a cotton swab or a toothbrush. [substeps] Or, try to find some woodland sticks or twigs with interesting shapes or thin stems.\nB. You can purchase paintbrushes at craft stores, or you can ask an art store or art supply store if they have them. [substeps] You'll need a sharpie palette, a spoon or palette knife, plates, brushes, and wooden dowels (any type will work).\nC. You will need : [substeps] Nail polish in the color (s) of your choice fine tipped nail brush or paintbrush clear topcoat [title] Paint your nails and let them dry. [step] Start by painting your nails in the color (s) of your choice.\nD. You can purchase a variety of craft tools from a craft store for a relatively cheap price at around $6 or less. [substeps] The best tools to use for this project is a hammer, mirror, paint brush, glass and a water bottle (make sure your bottle is clean and can be reused).","prompt":"[header] How to create star nail art [title] Gather your materials. [step] Painting simple stars on your nails is much easier than you might think. You just need the right tools to do it.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4937,"error":null,"output":"C"}
{"question_id":"hellaswag:00006998","item_index":16,"attempt":0,"prompt_hash":"6730c6669dfefc11330a42029a669714ef3ae4af02b34c65afeac6de96e9f9ed","question":"[header] How to choose dairy free snacks [title] Read the food label. [step] If you're looking to avoid a particular food or food group like dairy, you'll need to learn how to read the food label on packaged goods. This will give you the information on whether or not a food contains dairy and is appropriate for you to eat.\n\nA. Find out what the ingredients are in each food/food group that you may be eating. [substeps] Some foods have labels for dairy so try and read the label yourself.\nB. [substeps] Do a google search for products that contain dairy and read grocery-store ratings to learn more. You'll also be able to figure out which foods are considered most beneficial.\nC. [substeps] Look at the allergen list underneath the ingredient label. If it says contains milk or dairy or made in a facility that also processes milk or dairy, this product most likely has dairy included in it or may have been contaminated with dairy products it was produced with.\nD. [substeps] Some baked goods that are named \" dairy \" will have a label indicating they are dairy free. You can also check the ingredients list on a packaged food section or go online to familiarize yourself with the ingredient list.","prompt":"[header] How to choose dairy free snacks [title] Read the food label. [step] If you're looking to avoid a particular food or food group like dairy, you'll need to learn how to read the food label on packaged goods. This will give you the information on whether or not a food contains dairy and is appropriate for you to eat.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4676,"error":null,"output":"C"}
{"question_id":"hellaswag:00005753","item_index":17,"attempt":0,"prompt_hash":"9fe5409674c4d633e7288623526d19641c02545ede155aa2e02a0a668a2cf8f9","question":"[header] How to be a domestic goddess [title] Set a daily routine. [step] There are a handful of chores you should do each day if you want to maintain a clean and clutter-free domestic heaven. Thankfully, these chores are quick enough to squeeze in even if you work a job outside of the home or have a houseful of kids to attend to.\n\nA. [substeps] Organized and set your hours so that people have the illusion of more \" domestic. \" set a consistent household schedule, eat and sleep at the same times each day.\nB. [substeps] To reduce your stress as much as possible, plan on accomplishing each of these daily chores during a certain time of the day. Organizing your time may help you relieve some of the anxiety you feel about getting everything done.\nC. [substeps] Hygiene is especially important if you begin dealing with housework on a daily basis. Do your laundry every day to keep things tidy and clean.\nD. These chores will serve as a foundation for the rest of your housework. [substeps] Be sure you are picking up every detail you need to know for each task.","prompt":"[header] How to be a domestic goddess [title] Set a daily routine. [step] There are a handful of chores you should do each day if you want to maintain a clean and clutter-free domestic heaven. Thankfully, these chores are quick enough to squeeze in even if you work a job outside of the home or have a houseful of kids to attend to.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4933,"error":null,"output":"B"}
{"question_id":"hellaswag:00005749","item_index":18,"attempt":0,"prompt_hash":"f018acf43f5fbb9514d95469fde04acfb74133120e5cb0d0e07542dc8a82a712","question":"[header] How to remove an application (game) off your facebook account [title] Log in to facebook. [step] Type in your username and password when prompted. [title] Locate the particular game/app that you intend to remove.\n\nA. [step] This will be under the app category under \" settings. \" on the left sidebar, you should find \" games \" under \" apps.\nB. [step] Find the game/app, and download it to your computer. If you find it, navigate to the game/app download page and download the app : [substeps] Games for cfu game.\nC. [step] Once you type in the name, you will be brought to the bottom of the list and must select it, otherwise the menu will close and you cannot see the game/app's icon yet. [substeps] The app icon is the middle portion of the app that is located to the right of the apple id card icon.\nD. [step] Some activities are alike but for those activities which are more restricted, some can be blocked. Remember that what you're viewing/listening to and wanting to view on facebook is all unique.","prompt":"[header] How to remove an application (game) off your facebook account [title] Log in to facebook. [step] Type in your username and password when prompted. [title] Locate the particular game/app that you intend to remove.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4678,"error":null,"output":"A"}
{"question_id":"hellaswag:00009186","item_index":19,"attempt":0,"prompt_hash":"ec454149f6777638a5b68dc88cbf6d186226409e1b961b453b0048290c11b879","question":"[header] How to make almond milk tea boba [title] Order or buy tapioca pearls. [step] If you do not have an asian market in your town, they can be found easily online by conducting a search engine inquiry. You may also find them at tea shops in some cities.\n\nA. [substeps] The tapioca pearls are not as exotic as the other types. Some countries and regions offer bulbasaur pearls, which are available for purchase through costco and zapius.\nB. Make sure to order whole \" using this fresh grind \" tapioca pearls as they contain ethylene and vitamin c, which act to make beets. [title] Cut the boba pearls into small pieces.\nC. [substeps] Tapioca pearls may even come with an ingredient called tapioca coal. Purchase kosher walnut kernel pearls if you want to make almond milk tea.\nD. [substeps] Do not buy quick-cooking boba, for best results. This tapioca is usually used as a thickener for soups, casseroles and stews.","prompt":"[header] How to make almond milk tea boba [title] Order or buy tapioca pearls. [step] If you do not have an asian market in your town, they can be found easily online by conducting a search engine inquiry. You may also find them at tea shops in some cities.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":8414,"error":null,"output":"C"}
{"question_id":"hellaswag:00005395","item_index":20,"attempt":0,"prompt_hash":"ae6b9e3940dcf431fd2c717d303c7e5d9249458d05d7d122bf195e3960546112","question":"[header] How to spot a gifted toddler [title] Take note of their curiosity. [step] Your child may be gifted if they are intensely curious about the world around them. If they show curiosity through touch, smell or verbally, they may be gifted.\n\nA. If they communicate with other children in a clear way, they may benefit from reading books or videos about plants, animals, cultures and people. If your child expresses curiosity by studying plants at home, it's possible that they're gifted.\nB. [title] Go to their classroom and observe them. [step] Give them a tour of the space.\nC. An gifted toddler can send the message to adults that their lives are exciting, something that can lead to increase interest in life. [substeps] A gifted toddler will have an obsession with valuing things and because of this childlike curiosity, they are prone to be very inquisitive about things.\nD. For example, see if they ask a lot of questions about the world. Examples include the following questions : [substeps] \" why is the sky blue? \" \" where did i come from? \" \" why are there homeless people? \" [title] See if they have an exceptional memory.","prompt":"[header] How to spot a gifted toddler [title] Take note of their curiosity. [step] Your child may be gifted if they are intensely curious about the world around them. If they show curiosity through touch, smell or verbally, they may be gifted.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":8414,"error":null,"output":"D"}
{"question_id":"hellaswag:00006311","item_index":21,"attempt":0,"prompt_hash":"4e74733eec8becef685ed956d4299a5e0fdf1999985acfc6051426eab6094db0","question":"[header] How to double a recipe [title] Write each of the ingredients on a piece of paper. [step] Chefs don't recommend scaling a recipe in your head. You should write out the amounts you need ahead of time.\n\nA. [substeps] If you have a copier, you may want to copy the original recipe and write in the margins, so that you have the instructions next to the ingredients. [title] Write down all of the vegetables, flour and meat products in 1 column.\nB. [substeps] To \" double \" a recipe, you need to produce a detailed ingredient chart that you can use during the process. Add together the ingredient base, texture and-if needed-a hacksaw.\nC. Having a thin target circle in your head allows to create a more balanced meal. [substeps] Include between 1.5 and 2.5 oz.\nD. Choose a single side of the recipe and put your cursor in the appropriate place. [substeps] Write out the quantity in the following order : 00/350/3 tb/1 cup.","prompt":"[header] How to double a recipe [title] Write each of the ingredients on a piece of paper. [step] Chefs don't recommend scaling a recipe in your head. You should write out the amounts you need ahead of time.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4678,"error":null,"output":"A"}
{"question_id":"hellaswag:00006284","item_index":22,"attempt":0,"prompt_hash":"ba72dd66dfc8ad22f4071891aa0df76afebce954733f0fd99060ddc17c91a88d","question":"[header] How to use antibiotics effectively [title] Take only antibiotics specifically prescribed for you. [step] The doctor chooses an antibiotic and the dosage based upon your state of health, your weight, and what microorganism (s) are causing your infection. This minimizes the risk of side effects.\n\nA. Do not take medication that has not been specifically prescribed for you and your medical condition. [substeps] Let your doctor determine the treatment plan.\nB. In addition, it keeps your doctor up to date on your medical history. [substeps] While antibiotics are very effective against various pathogens, they cannot effectively treat a variety of ailments.\nC. [substeps] Make sure to always rinse the medication away after taking it. If you go to a pharmacist, see them whenever it comes in.\nD. Avoid taking antibiotics for several days to two weeks, as they may cause your infection to grow out of control. Examples of specific antibiotic treatments that are frequently prescribed to you: the following : [substeps] Aspirin 10 days: probably recommended for infection with a bacteria that is known as keratitis.","prompt":"[header] How to use antibiotics effectively [title] Take only antibiotics specifically prescribed for you. [step] The doctor chooses an antibiotic and the dosage based upon your state of health, your weight, and what microorganism (s) are causing your infection. This minimizes the risk of side effects.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4680,"error":null,"output":"A"}
{"question_id":"hellaswag:00001709","item_index":23,"attempt":0,"prompt_hash":"d8f399e4d684885667e5eb7c3ece70c8c8c50c2ec3fc367d1290e61b9ddb60e8","question":"A large cat is walking down a brick pathway with a leash alongside a small white dog. as the walk\n\nA. continues, the cat stops and begins staring at a parked car with large red flames painted on the side.\nB. , the dogs nose brushes the leash.\nC. , the dog pushes the woman on its front legs.\nD. , the black and white perform various turns and tricks with the small white dog.","prompt":"A large cat is walking down a brick pathway with a leash alongside a small white dog. as the walk","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4680,"error":null,"output":"A"}
{"question_id":"hellaswag:00004737","item_index":24,"attempt":0,"prompt_hash":"e54df2fb8238772e79a851bc34ae0f99269b27242f46aa562826d2fe2d458d53","question":"[header] How to wear a beanie [title] Cover your whole forehead and ears for a classic look. [step] The front of the beanie should be just a little over your eyebrows. Leave the beanie a little loose on top and in the back.\n\nA. [title] Draw a line across your forehead to make a big wedge. [step] Add a few more lines so that you have some volume and flare.\nB. [substeps] The front of the beanie should be loose enough to hang over your eyes and to lengthen your neck length, but not so loose that it creates a tent or knot. This create the impression of a receding hairline.\nC. Push your bangs beneath the hat, especially if they are a little greasy or flat. [title] Roll the cuff of the beanie once for a warmer wrap around the ears.\nD. [title] Allow time for creasing before choosing a beanie style. [step] Look like you just rolled out of bed on an extremely warm autumn day, or you want to make your beanie look like you just rolled out of bed.","prompt":"[header] How to wear a beanie [title] Cover your whole forehead and ears for a classic look. [step] The front of the beanie should be just a little over your eyebrows. Leave the beanie a little loose on top and in the back.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4933,"error":null,"output":"D"}
{"question_id":"hellaswag:00005737","item_index":25,"attempt":0,"prompt_hash":"ac9feb18d4338c1a8cdba68370f6350bdf49a122394dabcbab6195436dbe1cc9","question":"[header] How to remove scars on legs [title] Know what scars you have. [step] Before you choose a treatment, it is essential that you know what scarring you are dealing with, as some treatments will only work for certain types of scars. You should always consult with a dermatologist before pursuing any treatment.\n\nA. This will give you different possible treatments for the scar. [substeps] Scars are covered with natural adhesion.\nB. The main scar categories include : [substeps] Keloid scars: these are large, growth-like scars that result from a wound that heals itself too aggressively. Keloid scars may grow larger over time and sometimes even return after treatment.\nC. Seeing a doctor as soon as possible is always the best policy. [substeps] Don't test any over the counter creams and remedies-they often have unpleasant side effects and you should avoid them entirely if you have a family history of scars.\nD. You may have been prescribed x-rays or pulsed light in one case, but then did not share that cost with others. [substeps] Warts are more difficult to remove and usually disappear before the most serious scars remain.","prompt":"[header] How to remove scars on legs [title] Know what scars you have. [step] Before you choose a treatment, it is essential that you know what scarring you are dealing with, as some treatments will only work for certain types of scars. You should always consult with a dermatologist before pursuing any treatment.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4678,"error":null,"output":"B"}
{"question_id":"hellaswag:00001112","item_index":26,"attempt":0,"prompt_hash":"2be765b01326d8dcd8b7d821ed10ad81ddb029e0d8d86dedf8f89dd9cde22d47","question":"Man is siting in a bed playing an harmonica and an acordion. Man is sitting playing instruments in a room. man\n\nA. is standing in the room with a harmonica in his hand.\nB. is sitting in a room and instruments are hanging in the back wall.\nC. is acorion talking to the camera.\nD. is carrying an accordion in a room.","prompt":"Man is siting in a bed playing an harmonica and an acordion. Man is sitting playing instruments in a room. man","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4681,"error":null,"output":"A"}
{"question_id":"hellaswag:00008846","item_index":27,"attempt":0,"prompt_hash":"5c2e3adc7e51722460da5ec196cc249f3338f1b6364cda6b472839423421d1c6","question":"[header] How to cheat proof a relationship [title] Make your relationship (and each other) a priority. [step] Simply make time for it (the relationship) and each other. Even if you have to write it down in a calendar, schedule times and dates to be together and connect or bond or be intimate with each other in every way, on all levels, and in every area possible.\n\nA. [title] Bring your partner home from work or school early. [step] You can save time, money, and travel by reminding your partner of the important things you care about them.\nB. Have a weekly date with each other, and make the most out of your days off together. Go out once in a while and stay in once in a while.\nC. What matters most to an ex are weekend weekends or dinner dates. [substeps] Let your cell phone unplug when you're not with your ex, when you've been together only for a few weeks or less.\nD. [title] Be upfront and non-judgmental. [step] Let in and out of conversations and see how they result.","prompt":"[header] How to cheat proof a relationship [title] Make your relationship (and each other) a priority. [step] Simply make time for it (the relationship) and each other. Even if you have to write it down in a calendar, schedule times and dates to be together and connect or bond or be intimate with each other in every way, on all levels, and in every area possible.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":4934,"error":null,"output":"A"}
{"question_id":"hellaswag:00007298","item_index":28,"attempt":0,"prompt_hash":"1a5090b0a22e4d3c3a197b7a537daebecfcfedc2441824f1c2bdf1b68c7156ad","question":"[header] How to become a wedding officiant in california [title] Find a church. [step] Consider what faith or denomination would best fit your beliefs, and what requirements that group sets for becoming an ordained minister. Sit in on a few services at local churches to see if their style and focus are a good match for you.\n\nA. [substeps] Think about the different types of church you would like to attend, and what you feel most comfortable attending. While these aren't always quite as important, they'll help keep you focused on your passions when you're allowed to attend.\nB. [substeps] Speak to your friends and families about their belief systems before choosing the church you want, or keep in mind that you might need to convert some services to accommodate families and clergy. It is important to find the best church for your faith, but do not rule out any protestant faith and any other denominations.\nC. [substeps] Once you find a church whose message resonates with you, seek out senior members of the church and speak to them about ordination via seminary school. Most seminaries have an established history with particular denominations, so senior members can guide you toward a school that will be the best fit for your beliefs.\nD. You'll look for the students at churches who perform the traditional ceremony and feel suited to an officiant. Alternatively, check into weddings recognized by others to see if that congregation is committed to treating a wedding like a marriage ceremony in itself.","prompt":"[header] How to become a wedding officiant in california [title] Find a church. [step] Consider what faith or denomination would best fit your beliefs, and what requirements that group sets for becoming an ordained minister. Sit in on a few services at local churches to see if their style and focus are a good match for you.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4677,"error":null,"output":"C"}
{"question_id":"hellaswag:00002326","item_index":29,"attempt":0,"prompt_hash":"f2816a117e2b7ada9456d5b68c07afaa3683fe3a571dec6901d72e0cef25861f","question":"A weight lifter powders his hands on a stage. He walks over to a barbell, and lifts it up. he\n\nA. raises it over his head before dropping it.\nB. throws the weighted weight with and and up.\nC. lifts it over his head, wiping his hands down.\nD. enters the gym, lifting the weight for the audience.","prompt":"A weight lifter powders his hands on a stage. He walks over to a barbell, and lifts it up. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4934,"error":null,"output":"A"}
{"question_id":"hellaswag:00007751","item_index":30,"attempt":0,"prompt_hash":"03fd76c045c77ab94736434256793c4aa94e3ddb484ea945550e9c2425a98e05","question":"[header] How to buy pads without being embarrassed [title] Buy several other items while purchasing pads. [step] Many people feel having only one purchase calls more attention to what they're buying. If you want to feel more discreet purchasing feminine products, buy several other items at the same time.\n\nA. [substeps] Buying pads with cover can serve as a buffer. The reason women don't like a name brand pad is that the name brand pad was designed to be oversized.\nB. Some people report sales up to $10 per item. [substeps] If you're seeing a big trend happening, don't hesitate to buy pads.\nC. [substeps] If you tend to go for heavy attention, don't purchase much gel pads. These are going to be heavier and less discreet.\nD. This can take some attention away from your pads. [substeps] However, be careful of the other items you add.","prompt":"[header] How to buy pads without being embarrassed [title] Buy several other items while purchasing pads. [step] Many people feel having only one purchase calls more attention to what they're buying. If you want to feel more discreet purchasing feminine products, buy several other items at the same time.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4681,"error":null,"output":"D"}
{"question_id":"hellaswag:00006842","item_index":31,"attempt":0,"prompt_hash":"0115aaccd7c58596e22182d436c45531892186925b042d6e4d7601aadce844fc","question":"[header] How to remove watch band links [title] Measure the watch band. [step] Before you set about removing any watch links, it is necessary to measure your watch band so you know exactly how many links you will need to remove. To do this : [substeps] Position the watch on your wrist exactly the way you intend to wear it.\n\nA. If you want it to look more comfortable and easy to take off, start by measuring the distance from the watch band to one of the holes on the side of your wrist. You will need to measure halfway up your wrist.\nB. When you are satisfied with the way it is positioned, turn your wrist over so that the watch band clasp side is facing up. With the watch band on your wrist, gather the slack in the band and hold the links together to simulate them being removed.\nC. On a regular watch band, it will just slide off without you opening your eyes. Use a tape measure to measure the distance between the top of the band and the back of your watch band.\nD. Place one end of a tape measure around the neck of the watch and measure the distance between the neck and the bottom. You will find this distance on either side of the watch band.","prompt":"[header] How to remove watch band links [title] Measure the watch band. [step] Before you set about removing any watch links, it is necessary to measure your watch band so you know exactly how many links you will need to remove. To do this : [substeps] Position the watch on your wrist exactly the way you intend to wear it.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":4678,"error":null,"output":"B"}
{"question_id":"hellaswag:00008349","item_index":32,"attempt":0,"prompt_hash":"1243fb6602b2e2afdd09fe1478e4ab991d24bb9f4a6c7283dbb449d5cca6ae40","question":"[header] How to sand drywall [title] Put down drop cloths. [step] When you sand drywall, you are going to get powder everywhere. If you've ever had a flour accident in the kitchen, then you know exactly how much mess fine powder can cause.\n\nA. Remove drop cloths and place them on the counter while you sand. [title] Cover the floor with drop cloths.\nB. Sprinkle drop cloths over the outside of doorways and a drop cloth over the floor to prevent dust from collecting there. Apply a thin layer of sealant between the drop cloths.\nC. Because you're going to be mixing up tiny pieces of drywall, you don't want to ruin the fun of this project. Just make sure it's clean and dry before you begin.\nD. You don't want to be cleaning that up from all over the room and the rest of your house, so start by putting drop cloths on the floor. Then, you would have to sweep or vacuum for days to get all the dust up.","prompt":"[header] How to sand drywall [title] Put down drop cloths. [step] When you sand drywall, you are going to get powder everywhere. If you've ever had a flour accident in the kitchen, then you know exactly how much mess fine powder can cause.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6018,"error":null,"output":"D"}
{"question_id":"hellaswag:00008972","item_index":33,"attempt":0,"prompt_hash":"1102d5a623127dd85cc3bc4ad6ac27bdab248a1c04da0958446a4ec230c9a26b","question":"[header] How to bring in and keep customers [title] Start with noticeable and recognizable signage. [step] You want your drive-by traffic to know who you are and what your business does. [title] Make any day an excuse for an event.\n\nA. [step] Be it a sale, an anniversary, or an open house, celebrate by promoting an event and opening your doors to the general public. [title] Offer an incentive to visit or refer your business.\nB. [step] Attract pay stations, electronic signage, and specific customers to come visit and talk about your business with you if it seems like a normal event. For example, you might simply stand by the door, open your back door when customers have come over, or let them check the mail before they arrive.\nC. [step] Instead of just telling your customers, \" i'm closing the bars at 6, but one of my friends moved to spain earlier and met us there, and i just can't wait for her there. \" try to use these times to share your journey with customers so they can build respect.\nD. [step] Avoid logos and slogans, baseball cards, \" do what i tell you \", \" get the views \", and anything that could be copyrighted or shared with other customers. Be sure to try not to look as though you're putting on a front by leaving out greasy products in highly visible places, like public bathrooms and coffee shops.","prompt":"[header] How to bring in and keep customers [title] Start with noticeable and recognizable signage. [step] You want your drive-by traffic to know who you are and what your business does. [title] Make any day an excuse for an event.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6015,"error":null,"output":"A"}
{"question_id":"hellaswag:00005613","item_index":34,"attempt":0,"prompt_hash":"6b4d0b9bbc07185705fe23d32f467dc4890aa931b98175b0b39e471d26d3ca98","question":"[header] How to draw blueprints for a house [title] Have a decent idea of your desired outcome. [step] Before starting on paper, it's going to help to get a basic idea of what the house is going to look like. It is important to know how many rooms are needed and how many floors are going to be put in the house.\n\nA. For example, if the house is just going to be in the basement, then a measurement should be just to ask for time. [title] Make a measurement using a tape measure and tape recorder.\nB. [substeps] Make sure you have put some thought into the facility and that the party rooms are lined up and no more than 6 volunteers are needed. Figure out both the average fee for a living room, a budget or any number of amenities.\nC. [substeps] Rooms can range from complex to cramped. It's best to get a general idea and plan to make your house totally bigger and more perpendicular.\nD. [title] For the perimeter, use a pencil to draw the outside walls of the house. [step] Try to take up the whole grid paper to get a larger scale.","prompt":"[header] How to draw blueprints for a house [title] Have a decent idea of your desired outcome. [step] Before starting on paper, it's going to help to get a basic idea of what the house is going to look like. It is important to know how many rooms are needed and how many floors are going to be put in the house.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6016,"error":null,"output":"D"}
{"question_id":"hellaswag:00001731","item_index":35,"attempt":0,"prompt_hash":"41e8d3655321ededef3becdffcc17e425c6709ba5d9ad0c569a784f8888dd19d","question":"A man comes and cuts a section of the grass with a hand tool. He cuts a few more sections. then he\n\nA. holds up the grass he has cut to the camera and walks away.\nB. cuts a few more parts to create the robot.\nC. cuts the portion with cutting tools.\nD. chunks the grass with his knife.","prompt":"A man comes and cuts a section of the grass with a hand tool. He cuts a few more sections. then he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6022,"error":null,"output":"A"}
{"question_id":"hellaswag:00000320","item_index":36,"attempt":0,"prompt_hash":"7d1989e729f08ce4a8166c175af3626503cb3c003f58142f49a85ae1e42ae86c","question":"A child sitting at restaurant table places his hands in a dessert. The child uses his hands to eat the food. the adult\n\nA. scratches his chin.\nB. hands the child something from a purse.\nC. takes his hands off of his causing the child to fall over again.\nD. keeps the child from spilling the food on the floor and helps feed him.","prompt":"A child sitting at restaurant table places his hands in a dessert. The child uses his hands to eat the food. the adult","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6022,"error":null,"output":"B"}
{"question_id":"hellaswag:00004536","item_index":37,"attempt":0,"prompt_hash":"2d5958046ff6080800fd19beded2212bf34431b3d061450670bc5ec30f05167d","question":"[header] How to rip dvd movies to smart devices [title] Download a program to defeat dvd copy protection. [substeps] Compare different products, read any disclosures carefully and look for non-biased consumer reviews so that you can pick the best product. Choose a product that has a free trial period so that you can try it out before you commit.\n\nA. When purchasing the dvd \" hard drive protection for a factory download \" option, you'll want to make sure you're making the right kind of motion. [title] Look on the side of packaging and sample it before you buy to ensure that it complies with the quality of your graphics.\nB. [title] Insert the dvd that you want to rip into your computer's dvd drive. [step] If you have multiple drives, select the drive that contains the dvd that you want to rip on your computer screen.\nC. [title] Double-click the dvd you want to rip. [step] Use the red link below the movie summary page to open a menu where you can choose a program from a list.\nD. For example, a one year trial package from 1650's amazon.com may be more good than one month trial package if it includes cool video capabilities, which they don't. Look for a mac licensed to have a dvd ripper.","prompt":"[header] How to rip dvd movies to smart devices [title] Download a program to defeat dvd copy protection. [substeps] Compare different products, read any disclosures carefully and look for non-biased consumer reviews so that you can pick the best product. Choose a product that has a free trial period so that you can try it out before you commit.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6015,"error":null,"output":"B"}
{"question_id":"hellaswag:00001665","item_index":38,"attempt":0,"prompt_hash":"5db69650a3872ad307f199aadea078d66bbcfb6eb19f3d7c7c1ffd82f8a93d3d","question":"A boy stands on a basketball court holding a ball. other players\n\nA. make baskets on the court.\nB. swarm the boy holding the ball.\nC. are chasing after the ball.\nD. approach in the background, then the boy in the court runs and gives a high jump.","prompt":"A boy stands on a basketball court holding a ball. other players","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6020,"error":null,"output":"C"}
{"question_id":"hellaswag:00003994","item_index":39,"attempt":0,"prompt_hash":"84f0baf3a23eef4f4174a2fb8de2f0128ba271122528305781a2336d81acd5d4","question":"[header] How to make vegan chocolate sorbet [title] Combine the sugar and the cocoa powder in a heavy-based saucepan off the heat. [title] Add the water gradually as you stir the mixture into a paste. [title] Keep adding the water until all the water is combined and the mixture is smooth.\n\nA. [title] Bring this mixture to the boil but boil it only until the sugar has dissolved and then reduce the heat. [title] Simmer for an additional 10 minutes.\nB. [title] Place the mixture into the saucepan and bring to the boil. [title] Cover the saucepan with a lid and cook it uncovered for 45 minutes on low heat until it reaches boiling point.\nC. [title] Remove the mixture from the heat and stir in the cornflour, baking soda, and salt until well combined. [title] Allow to cool at room temperature for two hours and then pour into an empty pan.\nD. [title] Place in a slow cooker and cook the chocolate. [step] Stir constantly to prevent lumps from forming.","prompt":"[header] How to make vegan chocolate sorbet [title] Combine the sugar and the cocoa powder in a heavy-based saucepan off the heat. [title] Add the water gradually as you stir the mixture into a paste. [title] Keep adding the water until all the water is combined and the mixture is smooth.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6014,"error":null,"output":"B"}
{"question_id":"hellaswag:00004269","item_index":40,"attempt":0,"prompt_hash":"55ff71c5bc00388d240ddfdcbdc128c6b5d643ae638be7af4eec4742101552db","question":"[header] How to gain wisdom [title] Try new things. [step] It's hard to gain wisdom when you stay in and do the same thing day after day. You get wiser when you put yourself out there and give yourself the opportunity to learn, make mistakes and reflect on the experience.\n\nA. If you tend to be on the inhibited side, work on cultivating an inquisitive spirit and the willingness to put yourself in new situations. Every time you experience something new, you open yourself up to the possibility of learning and getting a little wiser for having tried it.\nB. Chances are, you will become more wiser as you study, learn and be more active. [substeps] Try things that doesn't require your full concentration.\nC. Taking the initiative to do something new might not improve your mindset, or keep you from trying something new. Take the initiative to try new things, and spend your daily existence maximizing your wisdom.\nD. Find some new topics to do and then get out there and do something! Changing your stress levels can really help to, adapt and build your enthusiasm towards it all. [substeps] Try martial arts, in which you organize competitions and compete in the ring.","prompt":"[header] How to gain wisdom [title] Try new things. [step] It's hard to gain wisdom when you stay in and do the same thing day after day. You get wiser when you put yourself out there and give yourself the opportunity to learn, make mistakes and reflect on the experience.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6021,"error":null,"output":"A"}
{"question_id":"hellaswag:00001168","item_index":41,"attempt":0,"prompt_hash":"9721472585d170ad71a9f0530652a1c5fadfa3b435fec4d003e41d5e9d34fa21","question":"A female weight lifter walks up to a barbell. she\n\nA. squeezes it and then jumps over the bar backwards.\nB. lifts it up to her neck.\nC. lifts it onto her shoulder before lowering it to the ground.\nD. lifts it up to her chest before seated.","prompt":"A female weight lifter walks up to a barbell. she","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6019,"error":null,"output":"C"}
{"question_id":"hellaswag:00000421","item_index":42,"attempt":0,"prompt_hash":"3c52874ab7e4dd2b4af38450388ee06b3afe1a62605a0f75ddfec191fa4f44ce","question":"The words \"dathan ritzenhein nyc marathon 2010\" appear on a black screen. still\n\nA. images of pyramids in egypt are shown and then the video exposes the pyramids returning to the pyramids and returning back to the objects and pyramids they were a part of while the credits roll.\nB. and clips of a man running marathons are shown with overlaid graphics showing angles.\nC. picture, azog discus, with a fence bordering the screen.\nD. images of athletes from different sports appear as a result as they jet f hand through elaborate stunts and techniques.","prompt":"The words \"dathan ritzenhein nyc marathon 2010\" appear on a black screen. still","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6019,"error":null,"output":"A"}
{"question_id":"hellaswag:00009604","item_index":43,"attempt":0,"prompt_hash":"b18eab26918ff87640509e7800a0909302619b40b53c0ac3b564c7f349670b90","question":"[header] How to give yourself a facial [title] Pull your hair back from your face. [step] Use a hairband or hairpins to secure your hair away from your face, so your forehead is completely exposed. This way your entire face will benefit from your facial.\n\nA. [title] Apply a light moisturizer to your face. [step] A moisturizer will make your skin look softer and keep you from developing a sunburn.\nB. [title] Use your hands to smooth your face. [step] Use a comb to gently smooth your face to create a smooth texture.\nC. [title] Don't spray any of the hairspray from your head near your cheeks. [step] Slightly bend your elbow, and shoulder-width apart.\nD. [title] Wash your face with a gentle cleanser. [step] Use your favorite facial cleanser to get your facial off to a great start.","prompt":"[header] How to give yourself a facial [title] Pull your hair back from your face. [step] Use a hairband or hairpins to secure your hair away from your face, so your forehead is completely exposed. This way your entire face will benefit from your facial.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6016,"error":null,"output":"D"}
{"question_id":"hellaswag:00003268","item_index":44,"attempt":0,"prompt_hash":"6dc42100bab624c0183397cbe03efb64d9a368f3000fedcdf5b4ac5add2404b7","question":"[header] How to maintain inventory accuracy [title] Choose an inventory monitoring method. [step] The key to maintaining accurate inventory is having an established method of inventory tracking, and sticking to the method faithfully. There are generally two options for inventory tracking-system or manual.\n\nA. Inventory systems are generally software products that connect to your accounting system or cash register and update your inventory records as you make sales and buy new product. On the converse, you can also track your inventory manually by keeping an inventory log on paper or in an excel workbook, and by updating your inventory yourself periodically.\nB. [substeps] Manual (\" system \") units use the digital inventory interface (esl) units you select to track inventory accurately. You enable excel to automatically detect inventory if it is not saved to system system standards.\nC. If you live in a remote area, the most basic format for inventory tracking is to provide a digital log of all shipments that come in. However, many people are resourceful, and this may help keep track of prices, promotions, sales figures, and other transactions.\nD. Manual inventory tracking is best performed before you go to determine where to take inventory, due to the low likelihood of accidentally discarding stock stock items. [substeps] Both methods require you to set up a system so you can monitor your inventory inventory up to a certain point and then adjust your accounting requirements accordingly.","prompt":"[header] How to maintain inventory accuracy [title] Choose an inventory monitoring method. [step] The key to maintaining accurate inventory is having an established method of inventory tracking, and sticking to the method faithfully. There are generally two options for inventory tracking-system or manual.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6013,"error":null,"output":"A"}
{"question_id":"hellaswag:00006838","item_index":45,"attempt":0,"prompt_hash":"7ca91c4defb260eb13dabed2920a61c7b4d9dc85c661ea9221f5711be783f5f0","question":"[header] How to avoid peeing on the tampon string [title] Sit on the toilet seat, but don't pee yet. [step] If you are using a public toilet, you may wish to squat above the toilet, without actually resting on the seat. Otherwise you can use a seat-liner or tear several strips of toilet paper, lay them on the toilet seat, and sit.\n\nA. [title] Use a toilet brush to gently brush away any contaminant from the brush. [step] This is important so that the brush doesn't have an unsightly dislodge from the string.\nB. [substeps] Make sure you have pulled down your pants and underwear or lifted your dress or skirt before sitting down. Squeeze the muscles around your urethra (the opening through which urine exits your body).\nC. [substeps] If you have a notepad, a pen, or a sharpie pen, use those to jot down notes. [title] Take some time to conduct research on tampons.\nD. [substeps] If you don't feel comfortable doing this yourself, see someone who can do it. You might tell them, \" it's okay to look for toilet paper.","prompt":"[header] How to avoid peeing on the tampon string [title] Sit on the toilet seat, but don't pee yet. [step] If you are using a public toilet, you may wish to squat above the toilet, without actually resting on the seat. Otherwise you can use a seat-liner or tear several strips of toilet paper, lay them on the toilet seat, and sit.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6014,"error":null,"output":"B"}
{"question_id":"hellaswag:00000374","item_index":46,"attempt":0,"prompt_hash":"6d1b9d581059303bfceec5bcb948e915978f324e29539b86579b6981328b8290","question":"A boy is standing outside on top of ice fishing out of a small hole in the ice. The boy struggles to get the fish out and someone comes and takes it out for him. now there\n\nA. are two fish and one of them is focused on freezing and he waves at the fish.\nB. 's a woman standing inside the water watching him.\nC. 's a scoop of ice cream and he eats it and grabs a small shovel.\nD. are four men visible and they all begin to fish out of the ice pond.","prompt":"A boy is standing outside on top of ice fishing out of a small hole in the ice. The boy struggles to get the fish out and someone comes and takes it out for him. now there","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6018,"error":null,"output":"B"}
{"question_id":"hellaswag:00005239","item_index":47,"attempt":0,"prompt_hash":"a811a80975c20305230b7374a565da474adb9c3ae62663baca64e27557a915cd","question":"[header] How to wear cropped pants [title] Wear cropped pants that end between your calf and ankle. [step] Most cropped pants are known as ankle pants, capris, or \" clamdiggers. \" these pants are designed to hit you below mid-calf, where your lower leg begins to narrow.\n\nA. This is the most attractive part of your lower leg, and you should use your cropped pants to draw attention to it. [substeps] Cropped pants should not end at mid-calf, which is the thickest part of your lower leg.\nB. The top will hide your pants and provide a more flattering fit. [substeps] Bell bottoms, pleated pants, or other styles with flares are considered casual pants with ankle pants.\nC. [substeps] When picking this type of pair, make sure they end in a slim cut that is not too sloppy. Short leg pants should also be replaced with a pair of cropped jeans that hit at the hips or the thighs for a face-full picture effect.\nD. This versatility can help you create a slimmer cut for the top half of your pants. [substeps] Make sure that the high waist of the pants hangs below your ankle and reach your legs past the ankle.","prompt":"[header] How to wear cropped pants [title] Wear cropped pants that end between your calf and ankle. [step] Most cropped pants are known as ankle pants, capris, or \" clamdiggers. \" these pants are designed to hit you below mid-calf, where your lower leg begins to narrow.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6017,"error":null,"output":"A"}
{"question_id":"hellaswag:00006160","item_index":48,"attempt":0,"prompt_hash":"8575cc43316f5da0afbc418639cfdc4017e640ae24721c0d437b1fbe290d2a2f","question":"[header] How to treat flu in toddlers [title] Keep your child hydrated. [step] Children may forget to drink enough fluids when they are not feeling well. In addition, you child will lose water faster than usual if her body is producing mucus or if she is running a fever.\n\nA. Answer all the pediatrician's and physician's questions as well. Also, get your child more fluids when they are thirsty in order to help them lose water faster.\nB. Offer her plenty of liquids frequently and encourage her to drink even if she isn't thirsty. [substeps] Good drinks include water, juice, clear broth, or warm lemon water.\nC. Offer a hot water bottle at dinner, and when it is full, have your toddler drink a cup of water. [substeps] Take the bottle to your toddler to get his juices flowing.\nD. [substeps] Provide hydration every three to four hours, or whenever your child is thirsty. If you notice your child crying at bedtime and she does not drink enough, take her temperature every hour or so to ensure the child gets enough fluids.","prompt":"[header] How to treat flu in toddlers [title] Keep your child hydrated. [step] Children may forget to drink enough fluids when they are not feeling well. In addition, you child will lose water faster than usual if her body is producing mucus or if she is running a fever.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6015,"error":null,"output":"B"}
{"question_id":"hellaswag:00003883","item_index":49,"attempt":0,"prompt_hash":"958614ba5c5fd62ae1ca67650310408a0b4e717743350e7290d88238cd50f9dd","question":"[header] How to wear tights with a skirt [title] Wear polka dot tights. [step] Polka-dots are a go-to if you are going for a retro look-like a modern (but more covered up) pin-up girl. Go for tights with small polka dots, large polka dots, or polka dots in a variety of colors.\n\nA. You can also find tights that look almost like polka-dots, but are actually cute, little hearts. Polka-dot tights are casual enough for any occasion, but they're also pretty easy to dress up.\nB. [title] Wear neon tights with your skirt if you have that dreaded \" knee-high black \" look. [step] Neon tights are the epitome of brightly colored tights.\nC. For no-frills look, try to get a print that has polka dots on it. [substeps] It's a good idea to go for these on the seams too.\nD. [title] Choose full lengths skirts for formal occasions. [step] Go for full height skirts if you are planning on going for a formal affair or an event; you can get a \" sophisticated \" cut, shape, and make-up that fits an occasion perfectly and matches your outfit perfectly.","prompt":"[header] How to wear tights with a skirt [title] Wear polka dot tights. [step] Polka-dots are a go-to if you are going for a retro look-like a modern (but more covered up) pin-up girl. Go for tights with small polka dots, large polka dots, or polka dots in a variety of colors.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6017,"error":null,"output":"A"}
{"question_id":"hellaswag:00004399","item_index":50,"attempt":0,"prompt_hash":"8d96a0f2ca2eb29060c292edd6bb72a0f18f8c98eaba7582060c10fa37e588a6","question":"[header] How to have a garage sale [title] Gather items for sale. [step] Go through those boxes in the attic, shed, closet or garage and select items you can sell. Then walk from room to room in your home ,.\n\nA. A garage sale lets you see which items would be sold, through easy shopping, ebay, etc. This is also not too expensive, you don't have to spend very much, but it can cost around $20.\nB. Like selling at a yard sale, you want to be sure that you have enough for one day's sale. [substeps] Big boxes are ideal for garage sales.\nC. [substeps] It can be hard to part with things, even if you never use them. If you haven't used something for over a year, it's a good sign you won't miss it.\nD. To make your garage sale : [substeps] Buy lawn chairs, beds or couches. Store gardening supplies in boxes such as plastic zip ties, sunscreen, and more.","prompt":"[header] How to have a garage sale [title] Gather items for sale. [step] Go through those boxes in the attic, shed, closet or garage and select items you can sell. Then walk from room to room in your home ,.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6017,"error":null,"output":"C"}
{"question_id":"hellaswag:00004016","item_index":51,"attempt":0,"prompt_hash":"92b4882f04a8b0c906c22a833efbe64e8028573163a206c403948c37f181503d","question":"[header] How to make life fun [title] Look around for activities in your community. [step] When you are out running errands, keep your eyes open for flyers advertising social events that are going on. Many places around your town might offer a bulletin board where such events can be advertised such as supermarkets, libraries, universities, community centers, etc.\n\nA. As a way to show your community that you value the world there is a very large population around you. [substeps] You can also take a look at local amusement parks.\nB. [title] Be social by making a point of chatting or texting with people from your social circle. [step] There is nothing like a chat bubble inviting people to chat, so don't hesitate to strike up a conversation.\nC. [substeps] Encourage that people come together to hold events. Announce the purpose of the activity on message boards or on bulletin boards.\nD. [substeps] You can also search the internet for events in your area. Try searching for \" local events in [insert name of town].","prompt":"[header] How to make life fun [title] Look around for activities in your community. [step] When you are out running errands, keep your eyes open for flyers advertising social events that are going on. Many places around your town might offer a bulletin board where such events can be advertised such as supermarkets, libraries, universities, community centers, etc.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6014,"error":null,"output":"D"}
{"question_id":"hellaswag:00003486","item_index":52,"attempt":0,"prompt_hash":"00e7f72f875579cf1aebca8f069bdf62074e8cb213ed07d83b944e44619fa620","question":"[header] How to apply neulash [title] Wash off eye makeup if you're wearing any. [step] Saturate a cotton pad with eye makeup remover, and gently run it over the top of your eyelashes. Then, take the pad and gently wipe in a downward motion to remove any makeup on your lower set of lashes.\n\nA. [substeps] You don't need to do this for the rest of your lashes. Because neulash makeup often looks washed out, it can make your eyes appear to have blackheads or pimples.\nB. [substeps] If you don't have eye makeup remover, you can use baby shampoo as an alternative. [title] Make sure your eyelashes are completely dry before applying neulash.\nC. [title] Grab the edge of a silk or chiffon pillowcase. [step] Create a hand-drawn pattern by placing a scarf along a section of the pillowcase.\nD. Not all eye makeup removers work on your eyes. [substeps] Use a soft eye makeup remover if you have weak eyes.","prompt":"[header] How to apply neulash [title] Wash off eye makeup if you're wearing any. [step] Saturate a cotton pad with eye makeup remover, and gently run it over the top of your eyelashes. Then, take the pad and gently wipe in a downward motion to remove any makeup on your lower set of lashes.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6016,"error":null,"output":"B"}
{"question_id":"hellaswag:00004219","item_index":53,"attempt":0,"prompt_hash":"cfc1b3999886e590395b0cd73b905cc8d3e4cf8c33d8e702e93c7537731ddfb6","question":"[header] How to understand the causes of seizures [title] Study the causes of seizures and understand that they are the result of abnormal brain function. [step] No underlying cause for seizures can be found in about 70 percent of cases. Another 30 percent of seizures are found to be from underlying epilepsy.\n\nA. Another 90 percent of seizures can be caused by a fever, organ failure, or-rare cases-death. [title] If you are having seizures, use a dictionary to see which epilepsy you are having.\nB. Epilepsy can also cause hallucinations, altered states or longer-lasting periods of consciousness without severe symptoms. [substeps] Seizures are a condition that occurs only when the brain is injured and damaged, in which case all hallucinations are ignored or let go of.\nC. [title] Know that a seizure may develop as the result of a head injury or lack of oxygen. [step] At the time a person suffers a stroke, for example, brain tissue surrounding the site of the brain bleed becomes abnormally excited.\nD. It is difficult to rule out the underlying causes of seizures or form a clear diagnosis. It is hard to discern between seizures and normal brain function.","prompt":"[header] How to understand the causes of seizures [title] Study the causes of seizures and understand that they are the result of abnormal brain function. [step] No underlying cause for seizures can be found in about 70 percent of cases. Another 30 percent of seizures are found to be from underlying epilepsy.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6016,"error":null,"output":"C"}
{"question_id":"hellaswag:00002152","item_index":54,"attempt":0,"prompt_hash":"5f722490abe2de953fec30a16582590cf670f3b68da141bb3cf67175797c380d","question":"There's a young girl wearing a brown sweater playing the violin. She strings the violin with her left hand while she uses the bow stick with her right hand. she\n\nA. continues playing her tune on the violin with full concentration without pausing.\nB. hits the keys ferociously with both hands as she continues to play the violin.\nC. pulls the violin with her left hand.\nD. lifts one foot for balance on the violin.","prompt":"There's a young girl wearing a brown sweater playing the violin. She strings the violin with her left hand while she uses the bow stick with her right hand. she","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6317,"error":null,"output":"C"}
{"question_id":"hellaswag:00005291","item_index":55,"attempt":0,"prompt_hash":"5a219cd4fd5f1ccdc6472867495f86123c36e37daa7679751b97e9390a5a0a54","question":"[header] How to reheat ribs [title] Thaw your ribs that need to be reheated if applicable. [title] Preheat your oven to 250 degrees fahrenheit (121 degrees celsius). [step] If the oven is any hotter than this, there is risk of the rib meat shrinking and getting tough.\n\nA. [title] Lower your oven temperature to 450 degrees fahrenheit (212 degrees celsius). [step] Your meat is now ready to cook.\nB. Unplug your oven and let it cool before you continue. [substeps] If the weather is warm, you might want to turn off the oven immediately to have it cool down more or eliminate all direct heat from the oven.\nC. [title] Slather barbecue sauce over the rib meat on all sides. [title] Wrap the ribs to be reheated in 2 layers of foil.\nD. [title] Place the ribs on a cookie sheet, and allow them to cool for 2 hours. [step] You can skip this step if you prefer low-fat or moderate-fat style rib meat.","prompt":"[header] How to reheat ribs [title] Thaw your ribs that need to be reheated if applicable. [title] Preheat your oven to 250 degrees fahrenheit (121 degrees celsius). [step] If the oven is any hotter than this, there is risk of the rib meat shrinking and getting tough.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6316,"error":null,"output":"C"}
{"question_id":"hellaswag:00002622","item_index":56,"attempt":0,"prompt_hash":"a9b43dfb1cfb37b13f60bf8190e0aed6b2fa3c89ce048ff15146929bccf5fdec","question":"A group of kids and adults are trying to catch crabs in the rocks. a man\n\nA. is holding a hose to help them.\nB. pulls one of the kids into the ocean.\nC. helps them get out.\nD. holds up a crab to go into a container.","prompt":"A group of kids and adults are trying to catch crabs in the rocks. a man","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6315,"error":null,"output":"B"}
{"question_id":"hellaswag:00003889","item_index":57,"attempt":0,"prompt_hash":"c0b6f2c14dd067b2d4862a8f95926dc2802d52559de40050ba0baa054860f51d","question":"[header] How to take care of ladybugs [title] Purchase ladybugs. [step] If you have a garden or a greenhouse, you should consider adding more ladybugs to the environment. Ladybugs are very useful for controlling aphids.\n\nA. Aphids are tiny insects that can destroy vegetable and flower gardens. [substeps] Ladybugs tend to hibernate during winter months.\nB. Ladybugs can be sold as small boxes, pots, or bowls. [substeps] Ladybugs come in all shapes and sizes.\nC. You might try to keep ladybugs away from the garden. You can also buy ladybugs from a local store that sells insect products.\nD. Ladybugs can be especially helpful to keep livestock and other animals away as well as washing your vegetable and insect products. [title] Purchase the ladybugs' natural habitats.","prompt":"[header] How to take care of ladybugs [title] Purchase ladybugs. [step] If you have a garden or a greenhouse, you should consider adding more ladybugs to the environment. Ladybugs are very useful for controlling aphids.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6316,"error":null,"output":"A"}
{"question_id":"hellaswag:00003928","item_index":58,"attempt":0,"prompt_hash":"7cb7f083d877889022deccad87556c0f5b8433de5b3fd15d7f156798f8216c3b","question":"[header] How to help children sleep [title] Select a bedtime based on your child's needs. [step] You want to make sure your child gets an appropriate amount of sleep for their age. Know how much sleep your child needs.\n\nA. For example, children need 7.2 hours of sleep a night. [substeps] You may want to go for a seven-or eight-year-old standard, which requires only eight hours of sleep a night.\nB. Then, select a bedtime that allows your child to get an adequate amount of sleep each night. [substeps] Toddlers need 9.5 to 11.\nC. [substeps] For children 12-15 years old, for instance, about the same amount of 7 to 9 hours of sleep every night is optimal. For under seven children, about 11 to 11 hours of sleep a night is ideal.\nD. If they are really young and need 3 to 4 hours of sleep per night, you want to make sure they get the recommended amount. [substeps] A seven year old should sleep 8 to 10 hours per night.","prompt":"[header] How to help children sleep [title] Select a bedtime based on your child's needs. [step] You want to make sure your child gets an appropriate amount of sleep for their age. Know how much sleep your child needs.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6315,"error":null,"output":"B"}
{"question_id":"hellaswag:00002719","item_index":59,"attempt":0,"prompt_hash":"72eaf3a6011815911389bf9988c80ec97c6dd367e13f7b0fd767fdfbeb470bd9","question":"The man grabs the fish from the ice and puts it right back into the hole he just pulled it out from while it's still attached to the hook and his wire. the man\n\nA. then holds the fish and pushes the fish away from the camera.\nB. then begins to pull on the fishing wire, pulls his pole up and and down a few times, and then adjusts the camera.\nC. continues to fish around and takes off the camera to show himself off.\nD. seems to have a problem with different fish and spins the haft of the reel between his fingers as he tries to find the right hole to pull him out.","prompt":"The man grabs the fish from the ice and puts it right back into the hole he just pulled it out from while it's still attached to the hook and his wire. the man","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":9808,"error":null,"output":"A"}
{"question_id":"hellaswag:00000438","item_index":60,"attempt":0,"prompt_hash":"be81a81835e4d6afde07e2f941b64e3fc996906cac3c2e297d8698bb6f1f6240","question":"A man is shown speaking to the camera and holding up various tools and products. he\n\nA. begins cutting a piece of fruit and measuring it with several bottles.\nB. scrubs down two surfboards and then wraps a towel around the top.\nC. stands in front of a weight bar and counts down.\nD. begins taking off a bike's handlebars and moving them around.","prompt":"A man is shown speaking to the camera and holding up various tools and products. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6328,"error":null,"output":"D"}
{"question_id":"hellaswag:00004799","item_index":61,"attempt":0,"prompt_hash":"8c440ce2b32d86559b816c657888174d2d22d5d826b877ce95202ecc7610d044","question":"[header] How to keep a waist cyncher from rolling up [title] Stretch the cincher before you put it on for the first time. [step] Pull both ends several times before trying to wrap the cincher around your waist. While the cincher should feel a little tight on you, you should be able to fit into it snugly.\n\nA. [substeps] Gather all three loose ends in your hands. One of the loose ends will give you the illusion of the cincher skipping around your waist instead of climbing up to your waist.\nB. [substeps] If your hips are in an uncomfortable position on the cincher, keep stretching the cincher by pulling at both sides. Avoid pulling at the front cincher, which draws the material along the inner thigh.\nC. [substeps] If your cincher has hooks, you should be able to attach the hooks one at a time in front of your waist. If you can't, stretch it a little more.\nD. Make sure both ends of the cincher are not long enough to touch, as a better cincher is going to roll up on you later. [title] Purchase a chest cincher for less success.","prompt":"[header] How to keep a waist cyncher from rolling up [title] Stretch the cincher before you put it on for the first time. [step] Pull both ends several times before trying to wrap the cincher around your waist. While the cincher should feel a little tight on you, you should be able to fit into it snugly.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6327,"error":null,"output":"C"}
{"question_id":"hellaswag:00008079","item_index":62,"attempt":0,"prompt_hash":"14b0242ab2dd0784f073b25e3fac7663c638ff3e0b3e71de72e1e004a76761e3","question":"[header] How to accept a marriage proposal [title] Don't ruin the surprise. [step] If you believe that your partner plans to propose to you, give him or her the chance to carry out a plan. Don't try to rush the process along.\n\nA. Let him or her take the lead, but try to appear as engaged as possible as you challenge your partner to compromise or change his or her mind. Avoid waiting until the timing is perfect to truly ask.\nB. Be thoughtful and allow your partner to decide the next level of urgency (the best time for you and for your relationship). [title] Have your heart set on the proposal.\nC. Don't give any indication that you expect it. [substeps] If you find out for certain that your partner is planning to propose, you can take two approaches.\nD. Waiting for a truly spontaneous proposal to come to fruition is unheard of. [substeps] Ask your partner what they'd want if you were the lucky one and they were your best friend.","prompt":"[header] How to accept a marriage proposal [title] Don't ruin the surprise. [step] If you believe that your partner plans to propose to you, give him or her the chance to carry out a plan. Don't try to rush the process along.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6327,"error":null,"output":"C"}
{"question_id":"hellaswag:00004933","item_index":63,"attempt":0,"prompt_hash":"bfd442b9c281d86bbfc2ac7a58bc2f8318d5bf1a60a588e8c53d83027f98cb46","question":"[header] How to take care of pearls [title] When you put them on, wear them after you're done with everything. [step] When you're taking everything off, take them off first. As an organic gem composed of calcium carbonate, pearls are more vulnerable to the chemicals found in cosmetics, hair spray, and perfume.\n\nA. If you have 1 large pearl, now would be the time to invest in 10 more, within the first 20 years. Just like any other stone, pearls are sometimes exposed to general pollutants, which mean they're more susceptible to infection and, less importantly, to degradation.\nB. Dress, style your hair, apply your make-up, and spray on your perfume before you put on any pearl jewelry. [title] Limit your use of pearl rings and bracelets.\nC. Stock a small jar and use old pearls as you find them, because they can keep from breaking. A spray bottle should be sufficient, but if you'd prefer to use a porcelain one, stock your applicator brush in a medium-sized jar.\nD. Generally, try wearing your pearls after you've finished bathing. [substeps] Those with olive oil removed earlier, and those without more exposure to the chemicals, often achieve a gloss that naturally helps conceal silver.","prompt":"[header] How to take care of pearls [title] When you put them on, wear them after you're done with everything. [step] When you're taking everything off, take them off first. As an organic gem composed of calcium carbonate, pearls are more vulnerable to the chemicals found in cosmetics, hair spray, and perfume.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6328,"error":null,"output":"B"}
{"question_id":"hellaswag:00008096","item_index":64,"attempt":0,"prompt_hash":"5292d1813930ed4c7f12da9e72784906d97724465aa802bce16162a06ad51ee5","question":"[header] How to make a flagpole [title] Cut a piece of pvc to your desired flagpole height. [step] For this flagpole, four to about seven feet high is best. Within that range, you are free to choose the exact height.\n\nA. [title] Cut two lengths of pvc pipe to your desired height. [step] Use a friend's scissors, or any old fishing line that has been used to cut the pieces down to a length that you are comfortable using.\nB. Buy the pvc and either have the store cut it to length or measure and cut it yourself. Use a hacksaw to cut the pvc.\nC. Line the pvc with a tape measure or kaxed plastic sheet before cutting. The jig here is simply to measure and mark the radius of each pole plus 12 inches (30.\nD. It would be good to reserve a copy of your measurements for the sake of accuracy. [title] Orient a stick or dowel at the height you need.","prompt":"[header] How to make a flagpole [title] Cut a piece of pvc to your desired flagpole height. [step] For this flagpole, four to about seven feet high is best. Within that range, you are free to choose the exact height.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5636,"error":null,"output":"B"}
{"question_id":"hellaswag:00002932","item_index":65,"attempt":0,"prompt_hash":"1812597c7df276def5585d75dec95893f3939412898aee5f14719a346a87a5d3","question":"A bunch of men are inside a green colored gym. They are lobbing a tennis ball back and forth. they\n\nA. are engaged in a game of tennis.\nB. bounce the ball up and down on the trampoline as they go.\nC. hit the ball up high into the air.\nD. knock balls off the hoops into the spectators.","prompt":"A bunch of men are inside a green colored gym. They are lobbing a tennis ball back and forth. they","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5640,"error":null,"output":"A"}
{"question_id":"hellaswag:00006738","item_index":66,"attempt":0,"prompt_hash":"cbd75a6743b231411c3c3ce5933be4afc5adc6e772b8b6a05dd1c66b0f3a2a72","question":"[header] How to make bead earrings [title] Gather your materials. [step] Making beaded earrings is quite easy. However, you will need to have some special materials ready before you get started.\n\nA. You will be making all the beads along the length of your tongue. To prepare, you will need : [substeps] 3 ounces of seeds 1 pound (240 g) of pearl or silver beads 3/4 pound (150 g) of flax coarse grit [title] Gather the beads and materials you need.\nB. This process is quite similar to making bead earrings, except that you'll need to get some beads of different colors and lengths, which can easily be threaded into the beads. This process will be a bit like making beads made from one or both panels at once.\nC. You will probably need to visit a craft store to find everything, or you can order your materials online. Before you begin, you will need : [substeps] Endpins (looks like a straight wire with a small button at the end).\nD. You will need a clear, sturdy, and comfortable work surface so that your jewelry won't fall off during the process. If you need to make any beads that you would like to craft and you can purchase beads at a craft store, do so.","prompt":"[header] How to make bead earrings [title] Gather your materials. [step] Making beaded earrings is quite easy. However, you will need to have some special materials ready before you get started.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5635,"error":null,"output":"C"}
{"question_id":"hellaswag:00002241","item_index":67,"attempt":0,"prompt_hash":"cfc86905b092b3ac1c019057bd3366f6320d3ebce52568db1be547030c0ec22e","question":"The man walks up and down, straight across the yard as he mows the lawn. The child watches calmly while resting on the man's back. the man\n\nA. continues to mow the lawn with his black lawn mower.\nB. stops mowing and the boy attempts to emulate.\nC. lays a sticker mascot on the ground then another before raking the grass into piles around the boy.\nD. takes his shirt off and begins talking while talking to the boy.","prompt":"The man walks up and down, straight across the yard as he mows the lawn. The child watches calmly while resting on the man's back. the man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5640,"error":null,"output":"A"}
{"question_id":"hellaswag:00008153","item_index":68,"attempt":0,"prompt_hash":"ae227d6ffb41b5c2094f4bb68974e374b8bd16742f08c26d911e3e61eb4cd2c3","question":"[header] How to relax using the feather method [title] Lie on your back and close your eyes. [title] Rest your hands on your belly. [step] You can also lay your hands on the floor, several inches from your body, with your palms facing up.\n\nA. Feet should be spread about shoulder-width apart, or whatever is most comfortable. [title] Take a moment to get comfortable so that you can lie still.\nB. [title] Bring your arms over your head and apply pressure gently to your buttocks using your palms. [title] Bring your legs up towards your head and pull them to your chest.\nC. [title] Spread your hands as wide as you can and spread your fingers wide. [step] You can use only your thumb and index or middle fingers.\nD. Remember that belly breathing can soothe you! [title] Breathe deeply. [step] For some people, breathing can reduce anxiety and stress.","prompt":"[header] How to relax using the feather method [title] Lie on your back and close your eyes. [title] Rest your hands on your belly. [step] You can also lay your hands on the floor, several inches from your body, with your palms facing up.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5636,"error":null,"output":"C"}
{"question_id":"hellaswag:00009080","item_index":69,"attempt":0,"prompt_hash":"3a189adceec8de3fafc85844540b28146f9833a227705b6e916310b274e02ac9","question":"[header] How to co parent with an uncooperative ex spouse [title] Discuss time arrangements. [step] If the court has not mandated your time arrangements, you need to have a discussion about them. You can't just leave the arrangement vague and expect it to work well.\n\nA. It's important to have a discussion about how the transition will affect the children and/or the work or professional life of both parties. [substeps] You also shouldn't let the court rule \" no \" even if your child wants to turn around and graduate with you.\nB. Bring up how the separation will affect the intended child's future. [substeps] Set out deadlines and define when the child will get the benefits that they are seeking.\nC. If you don't think you can decide on one alone, consider talking with a counselor to help work it out. [substeps] You need to discuss what nights the child will be spending with which parent, who's responsible for what activities, and who's responsible for which care-taking activities.\nD. Take your time and consider finding alternatives, even when you are in a time crunch. If your parents have not provided adequate time to consult with you on the matter, its important that you have time to talk about the matter.","prompt":"[header] How to co parent with an uncooperative ex spouse [title] Discuss time arrangements. [step] If the court has not mandated your time arrangements, you need to have a discussion about them. You can't just leave the arrangement vague and expect it to work well.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5635,"error":null,"output":"C"}
{"question_id":"hellaswag:00008110","item_index":70,"attempt":0,"prompt_hash":"a4d834bcafd02281d86c402659d4ffd021710c43ef8445ec81d77ef56c6f89a7","question":"[header] How to cold brew coffee [title] Buy good quality medium roast coffee beans. [step] The best coffee comes from beans that have been recently roasted, so try to get beans that have been roasted locally. If you can't find local beans, just find a coffee bean that you have enjoyed in the past.\n\nA. [substeps] If you were trying to grow beans, incorporate a few days' worth of roasts into your coffee to get the most benefits. If you're not sure if you like roasts, keep in mind that tannins tend to act as a sweetener, so try to mix up your coffee carefully.\nB. Ideally, this coffee will be spicy on the outside, but it should be processed well into the inside to create a bitterness. [substeps] You can also find gluten-free well-aged coffee.\nC. [substeps] If you have a grinder, buy coffee beans that have not been ground yet. Grinding your own will give you fresher and better tasting cold brew coffee.\nD. [substeps] Avoid organic coffee beans, which are less readily processed. Buy bags of fresh-ground coffee beans for better flavor.","prompt":"[header] How to cold brew coffee [title] Buy good quality medium roast coffee beans. [step] The best coffee comes from beans that have been recently roasted, so try to get beans that have been roasted locally. If you can't find local beans, just find a coffee bean that you have enjoyed in the past.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5635,"error":null,"output":"C"}
{"question_id":"hellaswag:00009968","item_index":71,"attempt":0,"prompt_hash":"b400ff4f44853a961e6b739c374690105849db0025477cf7a7745b37272f556e","question":"[header] How to know if you should work in customer service [title] Determine if you have patience. [step] Perhaps one of the most important personality traits for customer service representatives is patience. Patience, which is defined as the ability to deal with delay and complications without problem, is critical for customer service representatives.\n\nA. Patience is important for a number of reasons. Consider if you could handle : [substeps] Staying on the phone for long periods of time.\nB. This means that you should work patiently, giving your undivided attention to your customers and staying on task when others need you. When you are more engaged in the work of your customer service representatives, you are more receptive to the help and guidance that comes your way.\nC. Knowing if you are patient and attentive will significantly eliminate your fear and help you understand the work in front of you. [substeps] If you are a dedicated customer service representative, you should encourage the people in your organization, too, to have patience.\nD. Though it is a positive trait, people can express a lack of patience when pressed for time. It is necessary to be punctual and assure your employees that they can communicate with you without interruption.","prompt":"[header] How to know if you should work in customer service [title] Determine if you have patience. [step] Perhaps one of the most important personality traits for customer service representatives is patience. Patience, which is defined as the ability to deal with delay and complications without problem, is critical for customer service representatives.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5636,"error":null,"output":"B"}
{"question_id":"hellaswag:00003591","item_index":72,"attempt":0,"prompt_hash":"117b3aca229707315e024e2d7dc72e9b0f516b6441253943380af070fa3dd23e","question":"[header] How to make turkey sausage [title] Clean an area for mixing use. [step] Make sure your hands are thoroughly cleaned, as this will be your mixing tool. [title] Add 2 lbs (0.9 kg) ground turkey to the large mixing bowl.\n\nA. [step] Crush up 1/2 teaspoon 1 pinch of salt. [substeps] Note that this trick should be used for most kinds of meats, including beef and pork.\nB. [step] This is a typical amount for most turkey sausage recipes. [title] Combine all the seasonings in the small mixing bowl.\nC. [step] Make sure to scrape off any pieces of meat. [title] Stir the turkey together with a whisk or fork, until you achieve a small ball.\nD. [step] You can use dry tap water, but do not exceed the minimum amount of water as you do not want the ground turkey to rot. The better clean water you use, the better the flavour will develop as well as to absorb some of the smells.","prompt":"[header] How to make turkey sausage [title] Clean an area for mixing use. [step] Make sure your hands are thoroughly cleaned, as this will be your mixing tool. [title] Add 2 lbs (0.9 kg) ground turkey to the large mixing bowl.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5635,"error":null,"output":"B"}
{"question_id":"hellaswag:00004256","item_index":73,"attempt":0,"prompt_hash":"1192774d57e85789dcfb6691916b08fec37534fc4b7cd8ac9ae38eb0a6415506","question":"[header] How to sew a handy tool belt from an old pair of jeans [title] Find a pair of old jeans. [step] Cut the legs off, as these will be used to create your apron. [title] Take one leg.\n\nA. [step] This will be the main body of the apron. Cut so that it's approximately 17 inches (43.2 cm) in length.\nB. [step] Trace the shape of the leaving leg with the hole, doing this until it is smooth. Cut the seat in half along the line you have just traced.\nC. [step] Cut it so that it is only around four inches in length, as you will be sewing the belt piece to this length. [title] Sew about half-way up the legs to make the apron.\nD. [step] Turn it inside out and unstitch, always starting from one end. Be sure that the wound is facing the inner seam rather than the inside seam.","prompt":"[header] How to sew a handy tool belt from an old pair of jeans [title] Find a pair of old jeans. [step] Cut the legs off, as these will be used to create your apron. [title] Take one leg.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5632,"error":null,"output":"A"}
{"question_id":"hellaswag:00003201","item_index":74,"attempt":0,"prompt_hash":"687662096657db8776aa7abb3b26b00ad27a722b56a438e46f4949a76bb9d4e8","question":"A woman and a doctor are seen talking to one another and leads into the person putting dots on her eyebrow. He shows her in the mirror and then pierces the girl's eyebrow and puts a gem through. he\n\nA. then pierces the pinkie and then pierces the girl's right finger before sealing the jewelry on.\nB. continues to pierce the girl's eye and give her a shot.\nC. then cuts the top off and smiles to the camera.\nD. wipes off the blood and shows the finished piercing to the girl in the mirror.","prompt":"A woman and a doctor are seen talking to one another and leads into the person putting dots on her eyebrow. He shows her in the mirror and then pierces the girl's eyebrow and puts a gem through. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5635,"error":null,"output":"D"}
{"question_id":"hellaswag:00005525","item_index":75,"attempt":0,"prompt_hash":"ecfe800159cb805d0f7ccd39d90deabbc05b0819006d82c949c86e3e84fa4bc9","question":"[header] How to be thorough in your oral hygiene routine [title] Clean your teeth at least twice a day. [step] The best oral hygiene habit to get into is to brush your teeth at least twice everyday-once when you wake up in the morning, and again before you go to bed at night. If you do absolutely nothing else for your teeth, at least do this.\n\nA. For your other hygiene duties (like keeping your teeth clean, eating healthy, taking a shower, etc. ), do the following : [substeps] Gently suck on a paper clip or pencil.\nB. [title] Wait at least one hour after eating to brush your teeth. [step] When you eat and drink during a meal, the enamel on your teeth can soften temporarily.\nC. [substeps] Check the inside of your mouth to make sure you don't have anything in your mouth that is also stinky or bad smelling. If you do have some, don't put that into your mouth.\nD. [title] If you are going to be exercising, be sure to stretch, too. [step] At regular intervals throughout the day, you should stretch up and down to stretch out every muscle involved in exercising.","prompt":"[header] How to be thorough in your oral hygiene routine [title] Clean your teeth at least twice a day. [step] The best oral hygiene habit to get into is to brush your teeth at least twice everyday-once when you wake up in the morning, and again before you go to bed at night. If you do absolutely nothing else for your teeth, at least do this.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5633,"error":null,"output":"B"}
{"question_id":"hellaswag:00009223","item_index":76,"attempt":0,"prompt_hash":"b31d143440d356575df7c977bcea6be0e571b8d3be62d815cd564e245ad3a20d","question":"[header] How to move on when a loved one dies [title] Recognize and understand you may be angry. [step] You may be angry with the person who died. Remember that they didn't choose to die, but, however unfair it is, it was there time to move on, and they won't come back.\n\nA. [substeps] Grieving is not easy and feeling sad or angry after a loved one dies can make it hard to move on. Talk with your loved one if you feel as though you are unable to move on.\nB. Here are some underlying reasons why you might feel angry. [substeps] Whether or not your loved one wanted to die: take some time to consider why your loved one chose to die: it may have been a selfish thing to do or something he loved doing.\nC. Don't be angry with what ever you believe takes people out of the world (if you are religious). Remember that it is the same person that also brings people into the world, and it's the same person that gave you all the amazing memories you shared.\nD. Everyone makes mistakes, and you're no different. [substeps] When you are upset by someone die you know they made good decisions.","prompt":"[header] How to move on when a loved one dies [title] Recognize and understand you may be angry. [step] You may be angry with the person who died. Remember that they didn't choose to die, but, however unfair it is, it was there time to move on, and they won't come back.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5632,"error":null,"output":"C"}
{"question_id":"hellaswag:00009457","item_index":77,"attempt":0,"prompt_hash":"d2bcf4a63f2034b3d323b39fc5d0b37615f60f0b17e2582a4e05858aa6af1294","question":"[header] How to be independent while living in your parent's house [title] Create a communication system. [step] Communication is important between you and your parents to clarify when you need privacy or space. A simple, predetermined signal can denote when someone needs a moment of breathing room.\n\nA. [substeps] You might say something like \" i am feeling overwhelmed and need some privacy. \" this will communicate to them what you are feeling and what they can do to help you feel better.\nB. A simple signal will signify that someone needs a time to have privacy. This signals the person needs to take some time to clear his head, decide on where to go and what to do if someone is asking.\nC. Instead of telling your parents when you need to get out the door, explain that you're not ready for privacy or that you'd appreciate it if they let you call and text. [substeps] This \" phone \" communication is a sign of maturity and independence.\nD. The signal begins with a word or phrase like \" privacy, \" where you use your full verbal consent to inform your parents that you need privacy. You are legally responsible for whether you are comfortable sharing space when asked, and you need to be able to communicate verbally that you are independent when asked.","prompt":"[header] How to be independent while living in your parent's house [title] Create a communication system. [step] Communication is important between you and your parents to clarify when you need privacy or space. A simple, predetermined signal can denote when someone needs a moment of breathing room.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5632,"error":null,"output":"A"}
{"question_id":"hellaswag:00006059","item_index":78,"attempt":0,"prompt_hash":"f5a7838031440d52b3e4593b0ec1cedca14470048e2548805404597d0f88286b","question":"[header] How to bs your way through a college paper [title] Read the assignment requirements closely. [step] Consult the syllabus or your online course page for information on the assignment. See if it asks you to answer a specific question or explain research you've completed.\n\nA. You'll also review the worksheets needed to complete the assignment. [substeps] For example, if the assignment specifically asks you to write an answer to the question \" what happens when you write an answer...? \", think about taking a test that focuses on incorrect interpretations in a given course.\nB. Understanding what your professor requires can keep you on topic and maximize the time you have to write your essay. [substeps] Avoid sending the professor last minute questions about the assignment; it's a dead giveaway you did not do much work.\nC. If there isn't any help available, you'll have a hard time. [substeps] Contact your program's e-book or online course's assistance officer to find resources.\nD. [substeps] You may also want to think about if the course description clearly describes your work. If you didn't complete a course, then ask the instructor what the assignment description specifies.","prompt":"[header] How to bs your way through a college paper [title] Read the assignment requirements closely. [step] Consult the syllabus or your online course page for information on the assignment. See if it asks you to answer a specific question or explain research you've completed.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5633,"error":null,"output":"B"}
{"question_id":"hellaswag:00004327","item_index":79,"attempt":0,"prompt_hash":"33ec3c508ca58d678a83421463c47ea9a850f7ec42f1f6014ff65bda2cd5edbc","question":"[header] How to darken your skin [title] Spend time outdoors. [step] Spending time outdoors isn't only a great way to get a tan, it's also beneficial to your health. Being outside is the most natural and healthy way to tan your skin, which is why you'll get the most natural results.\n\nA. [substeps] Find a shady, sheltered spot to tan your skin, close to the flow of the beach. If you visit a pool or a stream, stretch to your feet and use your arms to support your body while tanning.\nB. [substeps] If you want to tan your skin, go for a walk in the sunshine. Rub sunscreen and moisturizing lotion over your skin as part of your suntanner or go swimming.\nC. Whether you're taking a walk, playing a sport, or having a picnic, you're still under the sun. [substeps] The appropriate amount of sun exposure is the best way to increase your vitamin d levels.\nD. You'll be able to breathe in fresh air, and you'll be in direct sunlight for hours on end. [substeps] This is also the best time to begin your veg tanning regimen.","prompt":"[header] How to darken your skin [title] Spend time outdoors. [step] Spending time outdoors isn't only a great way to get a tan, it's also beneficial to your health. Being outside is the most natural and healthy way to tan your skin, which is why you'll get the most natural results.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5634,"error":null,"output":"C"}
{"question_id":"hellaswag:00005942","item_index":80,"attempt":0,"prompt_hash":"5795632ea5a057f42a5ed53b5e19812dedb2d240b9682a2e0a2ffd66bfe5258f","question":"[header] How to become an au pair in spain [title] Consult online databases that list opportunities for working as an au pair in spain. [step] Alternatively, contract the services of a firm which will charge a fee in return for finding a position and providing logistical support throughout your stay. [title] Ensure that your language skills are adequate for your potential employers' needs.\n\nA. [step] Not ensuring fluent skills can be difficult, and employers may see you as weak in language such as spanish or french as a vulnerable you. A respectful, educated, articulate language can also be valuable.\nB. [step] Correspond with all potential employers before leaving for spain. [title] Agree to terms with your employer before accepting a position.\nC. [step] Speak english, french and spanish as many times as you can. A few words may not be enough if you are already an au pair but your english skills are desirable.\nD. [step] You can find a formal language program in madrid or milan in the region to apply for a job in this area. Also check to ensure that you both can speak english and french.","prompt":"[header] How to become an au pair in spain [title] Consult online databases that list opportunities for working as an au pair in spain. [step] Alternatively, contract the services of a firm which will charge a fee in return for finding a position and providing logistical support throughout your stay. [title] Ensure that your language skills are adequate for your potential employers' needs.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5634,"error":null,"output":"B"}
{"question_id":"hellaswag:00003947","item_index":81,"attempt":0,"prompt_hash":"c122c1dbb69669e6b353d81f4e643d00c28ef98321f9e66d24d55264a02011c6","question":"[header] How to lower your alimony payment [title] Read through your divorce decree. [step] Some divorce decrees include circumstances in which alimony automatically will be terminated or reduced. [substeps] If your decree includes such a clause and one of those circumstances has happened, your ex-spouse may be more amenable to the change.\n\nA. [title] Look at your alimony settlement in which your child is awarded less alimony. [step] A settlement does not promise to lower the alimony, rather you may end up paying your award up front.\nB. [title] Notice that there may be limited activities. [step] There are many reasons you might choose to reduce alimony payments for which there is limited activity.\nC. For example, your decree might specify that alimony payments will end if you lose your job. If you lose your job, the payments should automatically terminate according to the decree.\nD. To find out if you should meet your alimony order, go to this article: http: // www.alimony-less.com/how-to-meet-your-alimony-order-by-default /. [title] Agree to lower your alimony payment.","prompt":"[header] How to lower your alimony payment [title] Read through your divorce decree. [step] Some divorce decrees include circumstances in which alimony automatically will be terminated or reduced. [substeps] If your decree includes such a clause and one of those circumstances has happened, your ex-spouse may be more amenable to the change.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5634,"error":null,"output":"C"}
{"question_id":"hellaswag:00000850","item_index":82,"attempt":0,"prompt_hash":"c9b9da7aca545b6f58ba2c2b752fb421b54a48a80586da25d104a5af2a7e67d2","question":"This woman is showing her viewers how to make a chambord royale. first she\n\nA. grinds it and adds salt and baking soda.\nB. opens the bottle of beer that she had made and pours 1 perpatrick into the tiny glass with lime to continue drinking the beer.\nC. fills her shaker up halfway with ice and then she adds vodka and chambord and she puts one ounce of both of them and then she adds a little bit of pineapple juice and cranberry juice.\nD. demonstrates by mixing a mixture in a baking pan.","prompt":"This woman is showing her viewers how to make a chambord royale. first she","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5634,"error":null,"output":"C"}
{"question_id":"hellaswag:00005032","item_index":83,"attempt":0,"prompt_hash":"3d11da799a18f2bd5a422ec034729d846034549b9112ece016cf0f50da95a2c4","question":"[header] How to use the rear view mirror [title] Position your seat. [step] Before you adjust your mirror, you want to make sure that your seat is in the right position for driving. Move the seat forward or back, adjust the seat back, and move the seat up or down, if your seat has that type of adjustment.\n\nA. [substeps] You may need to experiment, but usually, most seats will adjust the mirror for you. For example, if you can adjust the wheel so that it faces slightly forward, that's a great move for a car with a full rear view mirror.\nB. [substeps] Most cars have the mirrors checked at their appropriate position prior to deployment. [title] Find your pre600 angle angle.\nC. [substeps] The outside parts of the body of the car, such as the pedals and headlights, should be aligned. Further, you should check your speedometer or your mirrors to see if you are in the right position.\nD. [substeps] It's important to adjust the seat so that you can easily reach the pedals of the car. Make sure your feet can reach the gas pedal, the brake pedal, and, if you have one, the clutch pedal.","prompt":"[header] How to use the rear view mirror [title] Position your seat. [step] Before you adjust your mirror, you want to make sure that your seat is in the right position for driving. Move the seat forward or back, adjust the seat back, and move the seat up or down, if your seat has that type of adjustment.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5634,"error":null,"output":"D"}
{"question_id":"hellaswag:00000802","item_index":84,"attempt":0,"prompt_hash":"8705571edef304c847f482111c529372f585c6230f4e65e8cdc3a0b05f458547","question":"The man examines the instrument in his hand. The person holds up the violin to his chin and gets ready. the person\n\nA. opens the violin case.\nB. makes the bow on his violin.\nC. finishes warming up and a hood is displayed.\nD. plays a song on the violin.","prompt":"The man examines the instrument in his hand. The person holds up the violin to his chin and gets ready. the person","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5632,"error":null,"output":"D"}
{"question_id":"hellaswag:00001822","item_index":85,"attempt":0,"prompt_hash":"3bfa1b096c1386a7a64c0481a5efb1d634e85c59ac00e7835512c9a9d625d816","question":"A close up of a box of ice cream is shown and leads into a bowl of ice cream sitting beside the box. the camera\n\nA. continuously pans around to all the scoops she and ice cream are using as well as the different scoops of ice cream the girl is using.\nB. leads into the bowl and turns towards a bearded man eating the ice cream and nodding in approvement.\nC. is captured close up as well as a woman grabbing the ice cream in her mouth.\nD. zooms in over the bowl and shows a mountain covered with pureed scoops.","prompt":"A close up of a box of ice cream is shown and leads into a bowl of ice cream sitting beside the box. the camera","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5633,"error":null,"output":"C"}
{"question_id":"hellaswag:00002656","item_index":86,"attempt":0,"prompt_hash":"046858533476638c68dfc563d2a24785261fa48e84b83ad3fdd2efeac59a3d83","question":"A camera pans around a person riding around on a horse. the person\n\nA. rides around on the horse and mows the lawn.\nB. gets off and continues riding down the road.\nC. is using various tools to horse up and down an obstacle.\nD. continues riding around the area and pauses to speak to the woman while she laughs and speaks to the camera.","prompt":"A camera pans around a person riding around on a horse. the person","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5368,"error":null,"output":"D"}
{"question_id":"hellaswag:00002401","item_index":87,"attempt":0,"prompt_hash":"c741725589b544f6237e2ff4160a3ca49892546b421460de50fd793caf114c84","question":"The young boy gets fitted for a pair of water skis on home patio. The young boy is pulled by the water ski instructor while on different types of floating devices. the child\n\nA. flips and turns multiple times against the wall while still attached to the water ski instructor.\nB. is then pulled behind a boat while on water skis.\nC. is pushed through the water by people and is carried by a boat down a river.\nD. rides a tube in the water skion the base of home patio.","prompt":"The young boy gets fitted for a pair of water skis on home patio. The young boy is pulled by the water ski instructor while on different types of floating devices. the child","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5368,"error":null,"output":"B"}
{"question_id":"hellaswag:00002002","item_index":88,"attempt":0,"prompt_hash":"334a4e40cf926862dc2d4ef08a6e8a536de63810523425033524222c5a9abfec","question":"A boy walk in the room from the left doorway holding a red stuffed animal sits down at the table then put animal away. a girl\n\nA. walk in the room front the right doorway and sits down at the table.\nB. came first and began to bungee jump very slowly and gracefully.\nC. is doing a karate trick on the table dodging the wall and counter chopped with her knife, then when she loses the trick the girl rest back on the table.\nD. shot a stuffed animal at a bed sitting on a brown armchair.","prompt":"A boy walk in the room from the left doorway holding a red stuffed animal sits down at the table then put animal away. a girl","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5368,"error":null,"output":"A"}
{"question_id":"hellaswag:00007903","item_index":89,"attempt":0,"prompt_hash":"d9a807b2eb4096c0319ea54197f83d90e09f2a508ba9643260e5685530f06dec","question":"[header] How to do a flat top [title] Establish how much hair to take off. [step] Talk with the person whose hair you're cutting and decide how long the top and sides should be. This information can help you decide which clipper blades to purchase (see step 3).\n\nA. [substeps] Do they want it thicker around the sides, or do they want skin showing around the head? How much hair do they want to leave on top? [title] Purchase a hair clipper from a local beauty supply store or online. [step] Three major brands are oster, wahl, and andis.\nB. Keep in mind that the more hair your hairstylist can cut off, the more space and extra work they'll have to put in. [substeps] As the hairdresser, you probably already have several ideas in mind when it comes to cutting your hair-but it's important to make sure you know them before you get started.\nC. [substeps] Although scissors can safely cut on hair, cutting too short can result in a slightly uneven ponytail. Stay away from using nylon hair detangling gels as they may rub against your hair, damaging the sections of hair you cut while cutting.\nD. Only use clippers that are larger than, say, a inch or two in length. Those with the smaller ones will not cut as deep as those with the bigger ones, so as to avoid damaging the cuticles.","prompt":"[header] How to do a flat top [title] Establish how much hair to take off. [step] Talk with the person whose hair you're cutting and decide how long the top and sides should be. This information can help you decide which clipper blades to purchase (see step 3).","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5368,"error":null,"output":"A"}
{"question_id":"hellaswag:00005956","item_index":90,"attempt":0,"prompt_hash":"241c0fabe4cef81382cb5bb678c2e8b774071d35576ed7e95cf687e57240160b","question":"[header] How to reformat windows 7 [title] Backup all your files, drivers and settings so that you can restore them later. [title] Find all your installation discs or product keys for the programs you want to keep so that you can restore them after the installation is complete. [title] Partition your hard drive.\n\nA. [step] For windows 7 you will need to partition all of your data, but this is optional as your computer needs your hard drive. [substeps] Right-click your drive and select partition all.\nB. [step] Partition your hard drive to remove your usb storage device from cd or dvd. [title] Finish the process using back up documentation if you wish to keep a backup device.\nC. [step] This means dividing the hard drive into parts and making the parts available to the os (operating system). [title] Click on \" start \" and then control panel.\nD. [step] This helps to prevent any lost drives. You can also patch any issues that you have with windows 7.","prompt":"[header] How to reformat windows 7 [title] Backup all your files, drivers and settings so that you can restore them later. [title] Find all your installation discs or product keys for the programs you want to keep so that you can restore them after the installation is complete. [title] Partition your hard drive.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5368,"error":null,"output":"C"}
{"question_id":"hellaswag:00001397","item_index":91,"attempt":0,"prompt_hash":"5e98e3e87a5a2afdda3284bb45c99f0e6ab913e5a39a662fe37ff58b2ddd9f72","question":"He rubs the soap together in his hands and massages into his face thoroughly. He then wipes his face off with a towel and speaking to the camera and pouring more lotion. he\n\nA. lathers his face into a lather while occasionally waving the item in the air.\nB. then leaves and talks to the camera again.\nC. continues shaving his facial hair and hanging it on the back of a chair.\nD. once again presses the lotion into his face and a recap is shown of his face.","prompt":"He rubs the soap together in his hands and massages into his face thoroughly. He then wipes his face off with a towel and speaking to the camera and pouring more lotion. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5368,"error":null,"output":"D"}
{"question_id":"hellaswag:00003537","item_index":92,"attempt":0,"prompt_hash":"9e9420092ecfa81439459b394d0f33badb6cdf694206ad8d23ab074ed7b78ea7","question":"[header] How to thaw frozen breast milk [title] Store milk in small portions. [step] Breast milk is only good for about 24 hours after thawing, so don't freeze more than one day's worth of milk in a single container. You can store your milk in special freezer bags or freezable bottles--two to four ounces is best for each container.\n\nA. Let the entire thing warm up before freezing. Avoid storing unused bottles, plastic bottles, or toasters.\nB. The size of the freezer bag depends on the amount of milk you're freezing. For example, a 50-gallon (325-liter) freezer bag could fit 2 jars of milk.\nC. [title] Refrigerate milk during cold weather. [step] After thawing, you want to store your milk on the counter at room temperature.\nD. [substeps] If you're using a container, choose a glass or bpa-free plastic container with a tight-fitting lid. If you're using bags, avoid using normal storage bags or bottle liners.","prompt":"[header] How to thaw frozen breast milk [title] Store milk in small portions. [step] Breast milk is only good for about 24 hours after thawing, so don't freeze more than one day's worth of milk in a single container. You can store your milk in special freezer bags or freezable bottles--two to four ounces is best for each container.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5367,"error":null,"output":"C"}
{"question_id":"hellaswag:00005469","item_index":93,"attempt":0,"prompt_hash":"4282bf75e1c1538bd6a34bd7a7bfba4075a7fe1aa35ef6d2b76ebace1c53ec40","question":"[header] How to apply moisturizer [title] Know the benefits of using facial moisturizer. [step] Moisturizers, in general, are great for your skin. When you apply a moisturizer to your face, it plumps up the skin and helps reduce the appearance of lines and wrinkles.\n\nA. It also protects the skin on your face from the various effects of weather, like heat and wind. And finally, and possibly most importantly, moisturizer increases the water content of your skin in order to keep it hydrated.\nB. Moisturizers in particular help keep your skin looking smooth and soft, to the point of preventing dryness. Keep in mind that over time you can use a more moisturizing powder, concealer, or make up application to keep your skin moisturized.\nC. It is important that you apply a moisturizer according to the product's instructions. [substeps] To apply the moisturizer, wet your face with lukewarm water.\nD. As a result, you'll have a smooth, aged looking face-without the unscarred scars. [substeps] See a dermatologist or skincare specialist for advice.","prompt":"[header] How to apply moisturizer [title] Know the benefits of using facial moisturizer. [step] Moisturizers, in general, are great for your skin. When you apply a moisturizer to your face, it plumps up the skin and helps reduce the appearance of lines and wrinkles.","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5368,"error":null,"output":"C"}
{"question_id":"hellaswag:00002482","item_index":94,"attempt":0,"prompt_hash":"3ca48a782f0362e41e8757fd2650d119a60dec34e8dd67189e6cbf877a043989","question":"A man in orange shorts exercises by jumping on a stack of workout mats. The man in the orange shorts works out on leg extensions on a leg extension machine. two men\n\nA. move the machine and we see the man jump off the workout machine.\nB. sit on a couch and talk about the workout side.\nC. work out on a rowing machine the man in the orange shorts works out on a different rowing machine.\nD. hovers over a blue area in a gym and spoke to the man who works out.","prompt":"A man in orange shorts exercises by jumping on a stack of workout mats. The man in the orange shorts works out on leg extensions on a leg extension machine. two men","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5367,"error":null,"output":"C"}
{"question_id":"hellaswag:00002745","item_index":95,"attempt":0,"prompt_hash":"fb2e5bdf3c46b20833c6fb41837928e718d2572095f3c498cd54091cf26f0779","question":"A shirtless man with a machete walks out into the front yard. He starts hitting the grass repeatedly over and over cutting the grass. instead\n\nA. of spinning around on the grass, he starts again and ends by walking away.\nB. of using a lawnmower he is cutting the grass with a machete.\nC. of stopping an gored up woman walks out from behind the house looking at the man.\nD. of cutting corners, the man starts to spread an shaped sheet out covering the entire yard.","prompt":"A shirtless man with a machete walks out into the front yard. He starts hitting the grass repeatedly over and over cutting the grass. instead","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5365,"error":null,"output":"B"}
{"question_id":"hellaswag:00008189","item_index":96,"attempt":0,"prompt_hash":"8b8413a34e8b1ead2f459f858cd495b7f9943e4142c9c0c9da73077eb9f45f6a","question":"[header] How to protect yourself from uv radiation indoors [title] Apply a protective film to windows. [step] Ordinary glass blocks \" short wave \" uvb light, but only blocks 50% or less of the more common \" long wave \" uva light. Add a uv-protective film to any windows that are not specially built to block uv light.\n\nA. [substeps] You can use compact plastic film for isolation, or layer bubble wrap for protection against uva radiation. By breaking down the bubble wrap, you won't risk the light harming the glass.\nB. You can purchase these after you go outside in the winter when the sun is out. [substeps] Just keep in mind uva radiation will unfortunately emit white light from above, so it will not be enough protection.\nC. This is especially important for windows near areas where you spend a lot of time, at home, at work, or in the car. The film blocks up to 99.9% of all uv radiation.\nD. Not every glass block may be covered in a uvb-protected film. For example, a window covered in a non-transparent film might still provide some uvb light.","prompt":"[header] How to protect yourself from uv radiation indoors [title] Apply a protective film to windows. [step] Ordinary glass blocks \" short wave \" uvb light, but only blocks 50% or less of the more common \" long wave \" uva light. Add a uv-protective film to any windows that are not specially built to block uv light.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5777,"error":null,"output":"C"}
{"question_id":"hellaswag:00003981","item_index":97,"attempt":0,"prompt_hash":"ad44c2e901720f521a29cf53c1a63d6f69b583959b4f1b6c558ba0872f155716","question":"[header] How to be more family oriented [title] Express your commitment to your family or significant other. [step] Being family oriented does not mean that you need to neglect everything else in your life to be with your family. However, it does involve an intentional shift in mindset to prioritize your family.\n\nA. If your family doesn't see your commitment toward them as a priority, then they may be more inclined to set aside time in their schedules for fun or activities. [substeps] Don't jump to conclusions about your time with your family or significant other.\nB. Rather than devoting all of your energy toward the child or other children, you can feel yourself spending much of your attention on one or two children. [title] Work on perfecting your standards for appearance.\nC. Incorporating a \" family \" role may help you feel less self absorbed. [substeps] Don't bring up the death of your significant other only because you feel compelled to.\nD. One way to increase your family orientation is to clearly convey your commitment to them. [substeps] You may say: \" i want you to know i really value your presence in my life, and i am here for our family.","prompt":"[header] How to be more family oriented [title] Express your commitment to your family or significant other. [step] Being family oriented does not mean that you need to neglect everything else in your life to be with your family. However, it does involve an intentional shift in mindset to prioritize your family.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5777,"error":null,"output":"D"}
{"question_id":"hellaswag:00002187","item_index":98,"attempt":0,"prompt_hash":"fb39163a3a7604affb1567046d41b2bfec589d2ea50e7b01452db3a83f23cc31","question":"One of the men throws a beer pong ball across the table, into a cup. The other man with him throws a ball across the table, too. two other men\n\nA. stand on the far right side of the beer pong table, and one of the men reaches to grab the cup where the ball fell into, and the other man bends down to catch a ball that has fallen on the floor.\nB. serve and a game of table foosball continues.\nC. play table tennis while a man in a red shirt watches their green cups.\nD. throw a ball across the table at the same time.","prompt":"One of the men throws a beer pong ball across the table, into a cup. The other man with him throws a ball across the table, too. two other men","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5791,"error":null,"output":"D"}
{"question_id":"hellaswag:00004745","item_index":99,"attempt":0,"prompt_hash":"4c6adecd52a673b94b24bdc659a5dbfddb87569ed9f59500c4ca3dc34c163ec3","question":"[header] How to choose toys that get kids moving [title] Supply your kids with a variety of balls. [step] Soccer balls, basketballs, and baseballs are excellent toys to get your kids moving. You can also purchase soft and squishy balls for younger kids and indoor use.\n\nA. [substeps] Keep in mind that kids can have many different toys at once. Try playdough, ramps, tennis balls, scooter, a handheld golf ball, and anything else that your kids like.\nB. You can find super soft balls for play toys, but often cheaper ones have a more smaller, heavier ball or ball to play with. [substeps] Buy small balls like ground balls, kiddie cups, and there are also miniature balls or balls made to collect playing material.\nC. Balls can be used to play traditional sports, or kids can use their imaginations and come up with a variety of games that will keep them active. [substeps] Balls can also help to develop kicking, throwing, running, and catching skills.\nD. You can also purchase sporting balls with the love of playing with them. You can buy play balls from amazon or miley's or other sporting goods stores, or have your kids construct their own for you.","prompt":"[header] How to choose toys that get kids moving [title] Supply your kids with a variety of balls. [step] Soccer balls, basketballs, and baseballs are excellent toys to get your kids moving. You can also purchase soft and squishy balls for younger kids and indoor use.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5783,"error":null,"output":"C"}
{"question_id":"hellaswag:00002803","item_index":100,"attempt":0,"prompt_hash":"b2015b0cb46524fd2c98d3a6096c13f7ab8837f4dcc0fd6515cd3fc89cc68028","question":"The man grabs the ball and takes a big step behind him and throws the ball. The man twists his body a few times as he holds the ball against his neck. the man\n\nA. does a 90 degree move around as he holds the ball against his neck.\nB. then extends both arms up and bows.\nC. then manages to candy cube the ball and throws it on the other side of the court.\nD. throws the ball to its opponent who easily catches it and throws it back to the matches.","prompt":"The man grabs the ball and takes a big step behind him and throws the ball. The man twists his body a few times as he holds the ball against his neck. the man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5789,"error":null,"output":"A"}
{"question_id":"hellaswag:00003823","item_index":101,"attempt":0,"prompt_hash":"61de95fbe9c1cf81971cf12c1e227b9cb73ca844b6b89892a04fe55fd7168584","question":"[header] How to travel around portugal [title] Time your journey. [step] Portugal is fairly small in terms of area. However, because of its shape (the coastline is 1 , 214 km (754 mi) long), distances to travel are substantial.\n\nA. Border guards see boats coming for foreigners from the global region, and stop as soon as you see one, especially on sunny and warmer days. If you leave portugal to undertake your trip, you will still need to adjust your tourist budget.\nB. Most migrating boats with casting nets will have a small space named latia in asia, and large boats with porto huts or scaffolding may be found in portugal, but require major differences to get where you want to go. Be patient, keep your boat at that size, and try to stay at least 4 days after casting out so that your home stays in portugal.\nC. The main town of portugal is portugal's capital district and loja. The rural area in portugal is about 36 km (43 mi) from portugal.\nD. Major points of interest (lisbon, porto, the algarve) lie along the coast. Plan at least a week to visit the most interesting spots; two weeks will be optimal! [title] Find an optimal time of year.","prompt":"[header] How to travel around portugal [title] Time your journey. [step] Portugal is fairly small in terms of area. However, because of its shape (the coastline is 1 , 214 km (754 mi) long), distances to travel are substantial.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5783,"error":null,"output":"D"}
{"question_id":"hellaswag:00004376","item_index":102,"attempt":0,"prompt_hash":"2026cbe0a9a1046cf50a61219383be5dcf2fd6352a788c8abcf7614fbd56e4ee","question":"[header] How to stretch a shirt [title] Wash your shirt in the washing machine or by hand. [step] Stretching out your shirt by hand will be much more effective if the fabric is wet. Use cold water when you wash the shirt to avoid causing your item to shrink even more.\n\nA. [title] Place your shirt in the dryer as a belt. [step] If you're confused about what kind of belt to use, ask an employee at a laundry supply store what a belt should look like.\nB. [substeps] If you're unsure how much to stretch, try a tape measure. If your shirt is only mildly stretchy, put a piece of tape over the article of fabric that you are stretching.\nC. [substeps] If you're hand washing, be sure to squeeze out excess water before stretching the shirt. Don't, however, wring the fabric.\nD. [substeps] Quick washing can also help a garment shrink faster. Avoid washing the shirt by hand unless it is an extra-large shirt.","prompt":"[header] How to stretch a shirt [title] Wash your shirt in the washing machine or by hand. [step] Stretching out your shirt by hand will be much more effective if the fabric is wet. Use cold water when you wash the shirt to avoid causing your item to shrink even more.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5787,"error":null,"output":"C"}
{"question_id":"hellaswag:00008816","item_index":103,"attempt":0,"prompt_hash":"dac5dbdeac5c89d1113466f4dd5c36e9e4d8482cc72eea5eeed7acea0799c5a5","question":"[header] How to eat on the go during pregnancy [title] Plan your meals and snacks. [step] Go into your week by planning ahead. Set aside one evening to look at your work schedule, so you can organize meals and snacks.\n\nA. Plan ahead by going into your bookstore and visiting a few books for lunch or dinner. Write out a for your evening meal ahead of time and see if you feel more motivated to eat or if you get hungry when you go in.\nB. This is especially important to do if you have a busier time of day or week when you might need to eat on the go. [substeps] For example, if you'll be running errands one day you might plan on making an oatmeal bowl the night before.\nC. This way you may also motivate yourself to indulge in your favorite foods while on the go. For example, if you like to go to the grocery store eat at home, grab a snack at home and set aside one of your meals that you can truly enjoy.\nD. This helps you to stay full and keep your energy up. [substeps] A meal at work can include a lot of' healthy food', such as veggies and lean proteins.","prompt":"[header] How to eat on the go during pregnancy [title] Plan your meals and snacks. [step] Go into your week by planning ahead. Set aside one evening to look at your work schedule, so you can organize meals and snacks.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5785,"error":null,"output":"B"}
{"question_id":"hellaswag:00001838","item_index":104,"attempt":0,"prompt_hash":"b0971821e262747d9aa061c153e5583f3e1dd828e6c1a9e8581a52027da6c4ef","question":"A kid lays under the tree putting things on it. The lady and a boy arrange the angel on top of the tree. the lady\n\nA. puts a chain around the angel's neck then tightens it.\nB. sets up a display table near the tree.\nC. puts flowers on the kid.\nD. raises her angel and puts decorations, then cuts the angel's wings with a machine.","prompt":"A kid lays under the tree putting things on it. The lady and a boy arrange the angel on top of the tree. the lady","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5787,"error":null,"output":"A"}
{"question_id":"hellaswag:00000113","item_index":105,"attempt":0,"prompt_hash":"fbcfaec888042af9ec9895ae8768ef73b396cbe16fa53a59aac3500828df16f7","question":"A group of men on donkeys play a form of polo on a dirt field. The camera pans to people watching the game. the camera\n\nA. moves to the women playing.\nB. pans between the playing couples.\nC. cuts to people riding horses on a dirt track.\nD. returns to the people riding the donkeys.","prompt":"A group of men on donkeys play a form of polo on a dirt field. The camera pans to people watching the game. the camera","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5787,"error":null,"output":"C"}
{"question_id":"hellaswag:00009868","item_index":106,"attempt":0,"prompt_hash":"1c7f9950fe3f156d1743b29d9b072174615992859f2d22bbaeca94347c8259c6","question":"[header] How to improve your personal integrity [title] Live with honesty and keep your promises. [step] Honesty is one of the most important aspects of integrity. If you are dishonest or misleading with others, you cannot live a life of integrity.\n\nA. Part of being honest with others also includes keeping any promises that you make. A broken promise is effectively a lie, and lying is inconsistent with living with integrity.\nB. When you find out someone has cheated on you, do not immediately respond with anger or frustration. Instead, take ownership for your behavior and let go of all pent up frustration.\nC. Instead, you must maintain honest behavior and morals. [substeps] Organize your thoughts in a logical fashion.\nD. Always be honest with your family, friends, teachers, coworkers, and partners. Some lessons to keep in mind include : [substeps] People who lie are at higher risk of injury and worry than those who don't lie.","prompt":"[header] How to improve your personal integrity [title] Live with honesty and keep your promises. [step] Honesty is one of the most important aspects of integrity. If you are dishonest or misleading with others, you cannot live a life of integrity.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5783,"error":null,"output":"A"}
{"question_id":"hellaswag:00008699","item_index":107,"attempt":0,"prompt_hash":"d2b2e4dc4bab6a2b746b260e94c20746d750dd6e9d9a587f65b54235722ad7ef","question":"[header] How to recognize insecurities [title] Evaluate your self-talk. [step] Do you pay attention to the constant conversation going on in your own head? Self-talk is either productive and positive or negative and detrimental to your well-being. Focusing on your self-determined negative qualities will keep you in a state of insecurity.\n\nA. [substeps] Try to think about your self-talk in a non-judgmental way. For example, instead of thinking, \" i am stupid, \" say, \" i am smart, \" or \" what social media can tell me about my intelligence? \" [title] Notice your self-esteem.\nB. In order to learn to effectively evaluate self-talk, you need to become mindful of any criticisms that arise. [substeps] You may make small changes in your behavior that have gotten the recognition of your self-talk.\nC. Harshly judging yourself does no good for anyone. [substeps] Avoid harshly judging yourself because it produces an unfair representation of you.\nD. [substeps] When self-talk ends, you may feel sad and depressed or anxious. Write down these feelings, especially anxious thought patterns.","prompt":"[header] How to recognize insecurities [title] Evaluate your self-talk. [step] Do you pay attention to the constant conversation going on in your own head? Self-talk is either productive and positive or negative and detrimental to your well-being. Focusing on your self-determined negative qualities will keep you in a state of insecurity.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5785,"error":null,"output":"C"}
{"question_id":"hellaswag:00009140","item_index":108,"attempt":0,"prompt_hash":"879dd65c70eeec709a57e7a0b000433f02662fe483f9181bca0340cfc681d846","question":"[header] How to cope with the loss of a loved one [title] Allow yourself to experience your feelings. [step] Losing someone you love is an emotionally devastating experience. In the aftermath of loss, you may find yourself grappling with a variety of feelings.\n\nA. Allowing yourself to experience these emotions can help you react in a more normal way. [substeps] It's probably best to try to maintain control of your emotions and accept your emotions.\nB. It's very important that you allow yourself to feel what you are feeling, even if it's bad. Suppressing your feelings is not a good idea in the aftermath of loss.\nC. Remind yourself of the good times that you have shared with this person. You may spend time with them, listening to their stories, and reminisce about a good time you had apart.\nD. You may take day-to-day guilt, regret, and grief over what you did to the person. You may find yourself frustrated with yourself over not surviving the loss.","prompt":"[header] How to cope with the loss of a loved one [title] Allow yourself to experience your feelings. [step] Losing someone you love is an emotionally devastating experience. In the aftermath of loss, you may find yourself grappling with a variety of feelings.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5781,"error":null,"output":"B"}
{"question_id":"hellaswag:00005090","item_index":109,"attempt":0,"prompt_hash":"940037e5ddea28ae738f1c14e73e951f2d59055aa632ec511f6f390d6f805383","question":"[header] How to use crystals for deodorant [title] Explore the benefits of using crystal deodorants. [step] The key ingredient for most crystal deodorants is salt, which when applied to the skin kills odor causing bacteria. It is a particularly good choice for those looking to avoid aluminum based deodorants.\n\nA. [substeps] Look for emollients containing this ingredient. Consider adding a dab of epsom salt to your deodorants before each use.\nB. This alone will kill off ionized sweat and other bad smell causing bacteria and reduce your body temperature. With this in mind, a more affordable choice of deodorant may be to apply it to your feet or underarms to treat sweat and bacteria, thereby defeating dirt and irritants.\nC. [substeps] Simply add one tablespoon of salt, massage it into the problem areas, then use warm water to rinse the salt off. Do not add too much salt or scrub the problem areas.\nD. [title] Consider using the stone form (either as a rock or a stick). [step] It is preferred by many because it is believed to be less altered by manufacturers than the other versions.","prompt":"[header] How to use crystals for deodorant [title] Explore the benefits of using crystal deodorants. [step] The key ingredient for most crystal deodorants is salt, which when applied to the skin kills odor causing bacteria. It is a particularly good choice for those looking to avoid aluminum based deodorants.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5784,"error":null,"output":"D"}
{"question_id":"hellaswag:00008271","item_index":110,"attempt":0,"prompt_hash":"c87a786c6dbbf8fa53032ac89220782dc64235a9df0a79717f04c08f8591afd7","question":"[header] How to act like a hipster [title] Avoid the mall. [step] Hipsters enjoy discovering new styles and brands. They also enjoy thrift stores.\n\nA. The mall is a great place to meet new people and expose yourself to new things. [substeps] Try to avoid the mall area as much as possible.\nB. A cluttered mall is not a match for a hipster. [substeps] Go for dresses that hit mid-thigh or higher, as this shows off your figure.\nC. Malls are seen as too mainstream and associated with authority. You don't want to buy mass produced products.\nD. If you want to get your name out there, then do so in different places. For example, you could head to a new york fashion shoestring in everyone's area, or buy your entire wardrobe from a thrift store.","prompt":"[header] How to act like a hipster [title] Avoid the mall. [step] Hipsters enjoy discovering new styles and brands. They also enjoy thrift stores.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5787,"error":null,"output":"A"}
{"question_id":"hellaswag:00006916","item_index":111,"attempt":0,"prompt_hash":"eb07e1b3869ba70edb8fd822a4cc2a193c4b8b0012edd7cb00537e109cd87f52","question":"[header] How to wear a denim vest [title] Choose a color. [step] Choose a color of denim that's right for you. Vests come in many shades of blue, a variety of blacks and even whites.\n\nA. You can mix both colors, or pick a color that's opposite or complementary. The first color is for example light blue or brown, while the second color is orange.\nB. Denim is typically available in several types of washes as well to really add a variety of color. Choose one that best suits your tastes and personality.\nC. The darker blue is best for winter, and the darker grey is best for summer. [substeps] If you do not want to invest in a vest, you can find one specifically made for the fall by nike.\nD. Regardless of what kind of color choice you purchase, steer clear of ones in the color range of that style. [substeps] For instance, a teal, cream, or black plaid vest would look great against a light yellow statement on a white tank top or black juicy skin short shorts.","prompt":"[header] How to wear a denim vest [title] Choose a color. [step] Choose a color of denim that's right for you. Vests come in many shades of blue, a variety of blacks and even whites.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5785,"error":null,"output":"B"}
{"question_id":"hellaswag:00005136","item_index":112,"attempt":0,"prompt_hash":"84b3c2213f17a9886721cc6ad4e5e6abf8d84deb03d7216bdb2e023563b4f708","question":"[header] How to get 360 waves [title] Ensure that your hair and scalp are in good condition. [step] The type of hair is not as much of an issue but the more prone your hair is to curling, the easier it is to create waves. If your hair is long enough to get curls, your hair will be long enough for waves.\n\nA. Make sure to start at your roots and work your way up to roughly manageable waves. [substeps] Make sure your hair and scalp are well-conditioned before you work on any new wave.\nB. Make sure your hair is healthy so it will grow nice and long. [substeps] If you have scalp problems, or are struggling with dandruff, remedy those problems before attempting waves with specialized scalp shampoo.\nC. [substeps] Washing your hair often is good because shampoo that is specifically made for your hair tends to clear hair more quickly. Try starting a dry shampoo habit and use just enough shampoo to maintain your hair shape throughout the day.\nD. If you have hair that is too short, your hair will curl and then your neck hair will curl. [title] Consider trying some hairspray and a curling iron.","prompt":"[header] How to get 360 waves [title] Ensure that your hair and scalp are in good condition. [step] The type of hair is not as much of an issue but the more prone your hair is to curling, the easier it is to create waves. If your hair is long enough to get curls, your hair will be long enough for waves.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5785,"error":null,"output":"B"}
{"question_id":"hellaswag:00006292","item_index":113,"attempt":0,"prompt_hash":"fba43982a127682fa7348a12c916cbc1aff790579da92cd75d2f59b88188d52f","question":"[header] How to extend your eyebrows [title] Sweep translucent powder over your brow bone with a clean brush. [step] A dusting of translucent setting powder with a fluffy brush will help the makeup stick to your skin. It will also absorb any excess oil that could cause your makeup to run.\n\nA. [substeps] If you have trouble with this, consider using a makeup sponge to rub translucent powder over your brow bone using the same swiping motion as you did with your brush. [title] Pull your brow hair into two buns out from under your ears and fold the sides inward at the center.\nB. [substeps] Translucent powder doesn't completely cover your eyebrow bones. It will actually help keep them clear and not cause them to wrinkle.\nC. [substeps] This technique prevents the makeup from sticking to your eyebrows and problem areas. You can apply translucent powder directly to the outer outer part of your brow bone to smooth it out.\nD. Dip your fluffy brush into the powder, tap the brush against the sink or counter to get rid of the excess, and then quickly dust a layer over both brows. [substeps] Alternatively, use a primer designed for use on your eyes to help your makeup stick.","prompt":"[header] How to extend your eyebrows [title] Sweep translucent powder over your brow bone with a clean brush. [step] A dusting of translucent setting powder with a fluffy brush will help the makeup stick to your skin. It will also absorb any excess oil that could cause your makeup to run.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5784,"error":null,"output":"D"}
{"question_id":"hellaswag:00006041","item_index":114,"attempt":0,"prompt_hash":"63cf6c6246f4994dc98815d6ecb05ecb5adb469ae3b15e79928b69c9a6fbb016","question":"[header] How to talk about suicide within the family [title] Be honest. [step] Unfortunately, suicide can carry a stigma with it that may prevent people from wanting to talk openly about it. If your family has experienced the suicide of someone close to you, though, you need to talk about it in order to heal.\n\nA. Say what you want to happen, but communicate clearly, in a way that is hurtful. [substeps] Don't share uncomfortable details or grievances in public, in front of family and friends, or with religious groups.\nB. [substeps] If you can simply explain that suicide cannot be taken seriously, people likely won't want to visit you any more. But if, despite your knowledge, the person before you explains that suicide cannot be taken seriously, most of your people in your family may want to talk about it.\nC. You should not talk about your loved one's suicide in front of others, who might interpret this as your opinion. You may feel embarrassed to tell about someone's suicide or even feel uncomfortable talking about it with your family members.\nD. You can have this discussion if you are all honest about what happened and how it is affecting you. [substeps] Encourage family members to say \" died by suicide \" or similar phrases instead of euphemisms like \" passed away that way \".","prompt":"[header] How to talk about suicide within the family [title] Be honest. [step] Unfortunately, suicide can carry a stigma with it that may prevent people from wanting to talk openly about it. If your family has experienced the suicide of someone close to you, though, you need to talk about it in order to heal.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5785,"error":null,"output":"C"}
{"question_id":"hellaswag:00002375","item_index":115,"attempt":0,"prompt_hash":"2487a045b8a2a5b92ed973bfe04b70f189dccabbe77cb97ae5b3d0edbcc6f002","question":"A green title page appears, with words showing that the video will be of a tenor saxophone player as well as the website to receive to lessons. A black male then appears with a green sweater and black hat playing the sax. the man\n\nA. bluffs on the drums and the video continues, at various points, to show individuals playing the instrument.\nB. continues to play and cheers, shakes hands with the man, throws a napkin, writes on the napkin and gives him a card.\nC. then lays the instrument across his lap and begins talking about what he is doing.\nD. jumps into a black and white video and temporarily stops playing, saying, \" big calibers \".","prompt":"A green title page appears, with words showing that the video will be of a tenor saxophone player as well as the website to receive to lessons. A black male then appears with a green sweater and black hat playing the sax. the man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5786,"error":null,"output":"C"}
{"question_id":"hellaswag:00006555","item_index":116,"attempt":0,"prompt_hash":"bb6fa728f5882a4916ebda14e91a7118febf808cf27ea191f7a98559f33f416e","question":"[header] How to deal with an extremely codependent family [title] Understand signs of codependency. [step] There are many signs. One of them is dysfunctional boundaries.\n\nA. Like boundaries with personal \" physical \" space, (ie. Standing too close to someone in an elevator) people in codependent relationships interfere with others by invading emotional space, allowing others to invade theirs, and having trouble distinguishing their own feelings, needs, and responsibilities from other people.\nB. By connecting perfectly easily with the people in your family and sharing your hopes and expectations for the future, you can respond consistently to many problems, including difficult schedules and interruptions in your daily routine. The two most common signs are : [substeps] Situations that should be avoided as a member of the household.\nC. Often the reason a codependent family members cannot manage their own life is that you break the family's traditions of routine and regularity. This is a particularly mature sign of dysfunctional behavior.\nD. During both pregnancies and pre-premenstrual cycles, a dysfunctional parent may try and repress you. In this case, you may need to cut them out of your life very quickly.","prompt":"[header] How to deal with an extremely codependent family [title] Understand signs of codependency. [step] There are many signs. One of them is dysfunctional boundaries.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5785,"error":null,"output":"A"}
{"question_id":"hellaswag:00005599","item_index":117,"attempt":0,"prompt_hash":"3fa5219c89d3266e001d50d23ecbb2473f54082f5e144da5ba335099bc9b759a","question":"[header] How to style your scene hair (for girls ) [title] Wash your hair. [substeps] Make sure to use both shampoo and conditioner to keep your hair clean and healthy. To be extra-thorough, you can try using a waterproof hairbrush while conditioning.\n\nA. Try not to over wash or overstyle your hair, as your hair will still absorb and the dye can become very dark. Choose a simple shampoo like dove to avoid damaging your hair and making your hair look oily.\nB. You can also dry your hair completely before dyeing it. It's less likely to get damaged.\nC. No shampoo or conditioner is needed if your hair is blonde. Use enough shampoo to keep your hair moisturized, as bleach can strip hair of its natural oils.\nD. If your hair is either wavy or curly, you might want to try using a straightening product as well. [title] Brush through your hair again after showering and towel dry it, but not too much.","prompt":"[header] How to style your scene hair (for girls ) [title] Wash your hair. [substeps] Make sure to use both shampoo and conditioner to keep your hair clean and healthy. To be extra-thorough, you can try using a waterproof hairbrush while conditioning.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5787,"error":null,"output":"D"}
{"question_id":"hellaswag:00005577","item_index":118,"attempt":0,"prompt_hash":"d2f3c46c2725af987ee79efb416d4a7ee7a88b243deda169e08f9cc1c613b3cd","question":"[header] How to draw a cool fashion design [title] Think of a style for your fashion. [step] Sketches are important to fashion designing because you want your idea on paper before you start doing the real thing. So, start sketching out your idea.\n\nA. If you find yourself \" ruminating \" and having not thought up the exact words to make your designs unique and original, you can at least use online sketches as a guide. [title] Pick a color you like and the look you want.\nB. It doesn't have to be perfect; it's a draft. Don't forget to add color! [title] Get a clear idea of what you want to draw in your head.\nC. Do it within the first few hours of a new project. Go back and think on the pencil sketch you bought at the store.\nD. Maybe a basic idea for an outfit, maybe a short one, maybe a more advanced idea for a traditional style. You'll enjoy the ideas! [substeps] Doodle the drawing walls with a pencil or by folding the paper back on one side and folding again; the right size and placement will come when you're drawing the design walls.","prompt":"[header] How to draw a cool fashion design [title] Think of a style for your fashion. [step] Sketches are important to fashion designing because you want your idea on paper before you start doing the real thing. So, start sketching out your idea.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7077,"error":null,"output":"B"}
{"question_id":"hellaswag:00000184","item_index":119,"attempt":0,"prompt_hash":"a7800ea1f2d8379b8e633a96f069e9d39da435366b6cc31f9bbc15e584780862","question":"We see a title card on black. A man is shooting basketball in a gym. We see a man dunk the ball twice. we\n\nA. see the ending title card on black.\nB. see a man flip and throw the ball.\nC. see man pull back the cup and drop a dart down the grass.\nD. see a yellow screen and title screen.","prompt":"We see a title card on black. A man is shooting basketball in a gym. We see a man dunk the ball twice. we","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7077,"error":null,"output":"A"}
{"question_id":"hellaswag:00009425","item_index":120,"attempt":0,"prompt_hash":"50210bd91c2e0816d3006e0a9488ce4b5503336b99986e509f496ce64cd8d5df","question":"[header] How to make jam muffins [title] Preheat your oven to 400 degrees fahrenheit or 205 degrees celsius. [title] Grease a muffin tray with non-stick greasing spray. [title] Whisk the dry ingredients.\n\nA. [step] In a large bowl, whisk the salt, flour, baking powder, and sugar until well combined. Use circular motions to get the ingredients to combine well together.\nB. [step] Place the wet ingredients in a large bowl and slowly add the wet ingredients at a time. Keep whisking until the batter forms to a smooth and creamy consistency.\nC. [step] In a large bowl, whisk the brown sugar, brown sugar, baking powder, and water. Transfer the wet ingredients to a separate mixing bowl.\nD. [step] By mixing the dry ingredients in with the brown sugar and baking powder, you're doing everything in the right order. You'll need to combine all the sugar in a saucepan over medium heat, mixing well each time.","prompt":"[header] How to make jam muffins [title] Preheat your oven to 400 degrees fahrenheit or 205 degrees celsius. [title] Grease a muffin tray with non-stick greasing spray. [title] Whisk the dry ingredients.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7077,"error":null,"output":"A"}
{"question_id":"hellaswag:00004578","item_index":121,"attempt":0,"prompt_hash":"f8049ae4099c4f22531ffa45b7480522dc60495f08f0cc12304798c405b7ccf1","question":"[header] How to deal with a teenage son [title] Make your expectations clear. [step] Your teen is less likely to push back if he has a voice in some of the rules. Sit down and have a talk with him about what each of you think is fair in terms of rules and consequences for breaking them.\n\nA. Post these rules in a common area so that they can be accessed easily. [substeps] The consequences should be appropriate for the rule that was broken.\nB. Agree on what the rules are so that he doesn't fall in line with some of the consequences. [substeps] Figure out your son's boundaries with a fair understanding of what does and doesn't matter.\nC. [substeps] Establish specific consequences for breaking certain rules. For example, start by asking your teen how much privileges he'll get when he's home.\nD. [substeps] For example, don't lead a teenager into doing something he won't enjoy, no matter how much his parents may say that it would help him better cope with school. Many teens are insecure about parental control, especially when things in a relationship revolve around expectations.","prompt":"[header] How to deal with a teenage son [title] Make your expectations clear. [step] Your teen is less likely to push back if he has a voice in some of the rules. Sit down and have a talk with him about what each of you think is fair in terms of rules and consequences for breaking them.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7077,"error":null,"output":"A"}
{"question_id":"hellaswag:00005512","item_index":122,"attempt":0,"prompt_hash":"88ee024dfc28e1ed1d2e47111f8656bc40c1f22d0cc80a78a37355b848279d03","question":"[header] How to use a microphone on a pc [title] Figure out if you need an external microphone. [step] Sometimes, it's text that gets the word across easier than if you talk. However, there are other times when talking gets the word across better than text-alone.\n\nA. [title] Use a built-in microphone in your pc, if your pc is installed with one. [step] Much like talking into a camera-microphone or telephone receiver, you can record sounds from your voice just by talking into it.\nB. [substeps] A large show might not require an external microphone. If you need to talk over hyper-large crowds, you'll need an external microphone.\nC. If you struggle with this, ask someone to point you in the direction of an external microphone, which is often easier than outside microphone displays. If you hear feedback on the microphone, it might come from the recipient of the text, as well as the printer if it's good reception.\nD. If this is the case, your brain can connect more effectively with the microphone (\" i'm narrating \"). [title] Consider getting an external microphone to use on your pc.","prompt":"[header] How to use a microphone on a pc [title] Figure out if you need an external microphone. [step] Sometimes, it's text that gets the word across easier than if you talk. However, there are other times when talking gets the word across better than text-alone.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7076,"error":null,"output":"A"}
{"question_id":"hellaswag:00003008","item_index":123,"attempt":0,"prompt_hash":"20f752c05ef4a1e590204fc1c98dd46533537be47614586957f02dc3f810b213","question":"A man is seen bending down before a set of weights with others watching him on the side. the man\n\nA. then lifts the weights up around his head and drops them down while others stand on side of the chair and watch.\nB. then takes the weights and lifts himself up while others do the same on the sides.\nC. lifts up the weights over his head.\nD. then lifts up the weight over his head and throws it down towards others.","prompt":"A man is seen bending down before a set of weights with others watching him on the side. the man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6864,"error":null,"output":"C"}
{"question_id":"hellaswag:00003302","item_index":124,"attempt":0,"prompt_hash":"0374a4c0b74376475531c6e31971aa77edc3aec009989de5e2e241fafcfdbd79","question":"[header] How to make a morning or night beauty routine [title] Decide what time to wake up. [step] It is important to wake up on the same time each morning, for beauty's sake! It is also important to go to bed at the same time, even on weekends. [title] Eat breakfast.\n\nA. [step] Morning beauty is, in fact, your wake up time. Let yourself indulge yourself in a tasty and big breakfast.\nB. [step] Breakfast is the single most important thing in life. Unless you prefer something, you're going to get hungry earlier everyday.\nC. [step] Studies show that woman who eat breakfast live longer, are fit, and look younger! [title] Brush your teeth. [step] Be sure to use mouthwash, and pop in a mint or some gum throughout the day for super-fresh breath.\nD. [step] Breakfast is the first meal of the day, and in general you wake up hungry ten minutes before you need to get up. This system takes care of you and keeps you on track throughout the day, but since it isn't that important a morning meal might be what you are looking for with your morning daytime beauty routine.","prompt":"[header] How to make a morning or night beauty routine [title] Decide what time to wake up. [step] It is important to wake up on the same time each morning, for beauty's sake! It is also important to go to bed at the same time, even on weekends. [title] Eat breakfast.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6864,"error":null,"output":"C"}
{"question_id":"hellaswag:00007765","item_index":125,"attempt":0,"prompt_hash":"6c0b8e629726a81be9ff389738002c0dc7cc8dc84888cb3507d800e105cfbd4a","question":"[header] How to do a voice over [title] Write out a script. [step] If you are doing commentary on something, like a youtube video, you should at least view the video several times to get a feel for what happens. For all other voice overs a script is essential.\n\nA. You need to know how long you have to talk, if you are responding to any characters or cues in the video, and what you're going to say. This script may change when matched with the final video, but you can never go wrong with a little pre-planning.\nB. Because you want your voice to be heard on all forms of video, you should try to write it professionally. [substeps] Do not write \" i'm in a fight scene \" or \" the other girl leading him around the room.\nC. All overs require large spaces, so the space needs to fit in between each voice one meter away from each other. Think of these as pathways to explore and move forward within a voice.\nD. ( make sure that your script is at least a mile away from your interview. ) [substeps] Write a script in black-and-white, or white-color, if you're script is not black and white.","prompt":"[header] How to do a voice over [title] Write out a script. [step] If you are doing commentary on something, like a youtube video, you should at least view the video several times to get a feel for what happens. For all other voice overs a script is essential.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6864,"error":null,"output":"A"}
{"question_id":"hellaswag:00003004","item_index":126,"attempt":0,"prompt_hash":"88e5191d4e93b5d752df7a68436c0dde1937432ae49ceaa3bdfd79f5532d5565","question":"The lady moves a shirt around on an ironing board. A lady is ironing a button down shirt. the lady\n\nA. puts the iron down.\nB. picks up a cloth and rubs the ironed shirt.\nC. takes two shirts and puts the worn out tee shirts.\nD. stands on the ironing board, and holds an iron over the shirt.","prompt":"The lady moves a shirt around on an ironing board. A lady is ironing a button down shirt. the lady","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6864,"error":null,"output":"D"}
{"question_id":"hellaswag:00006780","item_index":127,"attempt":0,"prompt_hash":"61278ee4049589dfb708e865a4f273b942fa3ad20f4690aedd28e7154d5e1e5e","question":"[header] How to get over someone quickly [title] Take some time to grieve, if you need to. [step] This is a normal part of the process of breaking up. It is usually not healthy to put a deadline on grieving, it's just something that \" is \", so be careful not to rush your grieving time simply because you want to be over this person quickly.\n\nA. [substeps] Losing a loved one is painful, because there are so many enemies and the loss of a friend is hard. When someone gets hurt, you may too.\nB. Remember that the person who hurt you is not your whole human self. And it is okay to grieve.\nC. [title] Make a decision that you're going to be just fine. [step] Whatever this person meant to you before the breakup, you are worth too much to keep feeling hurt for the sake of this person.\nD. [title] Give yourself time to grieve. [step] It will get easier and you will start to slow down and heal.","prompt":"[header] How to get over someone quickly [title] Take some time to grieve, if you need to. [step] This is a normal part of the process of breaking up. It is usually not healthy to put a deadline on grieving, it's just something that \" is \", so be careful not to rush your grieving time simply because you want to be over this person quickly.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6864,"error":null,"output":"D"}
{"question_id":"hellaswag:00003612","item_index":128,"attempt":0,"prompt_hash":"58cf57ea3b36b04babf39ba468a4678bd3a8e37049fac405023d2f352e900719","question":"[header] How to install entry doors [title] Remove the old door and the surrounding trim, molding, and door jamb. [step] You must remove everything covering the \" rough frame \" of the doorway. [substeps] The majority of doors for sale on the market today come pre-hung--that is, they are already mounted on a door jamb.\n\nA. If you were to install a new door through its original frame--the whole thing would be finished simply on the inside frame. Look for other ways to go about this.\nB. When installed correctly, they provide the tightest possible seal to keep out the elements. The door jamb is usually held in place with nails, screws, and caulk.\nC. For the best results, you should remove them, the plaster, wood floor trim and doormolding. [title] Slide inside the door and insert the new door.\nD. The good thing about plastic doors is that they are easy to remove. Some imperfections can be removed (nails are not necessary) with a phillips head screwdriver, or you can remove old door knobs by loosening the screws holding them in place.","prompt":"[header] How to install entry doors [title] Remove the old door and the surrounding trim, molding, and door jamb. [step] You must remove everything covering the \" rough frame \" of the doorway. [substeps] The majority of doors for sale on the market today come pre-hung--that is, they are already mounted on a door jamb.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7381,"error":null,"output":"B"}
{"question_id":"hellaswag:00000321","item_index":129,"attempt":0,"prompt_hash":"53cfb9f76480c48229b5ce93e185acef5b4076f101c356d319f996a48a60d978","question":"A man is first seen speaking to the camera with text questions being asked to him. He is then seen putting a cleaner on his teeth and taking a bit of food. he\n\nA. scrubs the dish vigorously in an attempt to get rid of the food in there.\nB. continues speaking to the camera while occasionally eating food.\nC. then smokes some smoke while looking off wildly into the distance.\nD. eventually is seen applying a gel and being shaved.","prompt":"A man is first seen speaking to the camera with text questions being asked to him. He is then seen putting a cleaner on his teeth and taking a bit of food. he","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7386,"error":null,"output":"B"}
{"question_id":"hellaswag:00006807","item_index":130,"attempt":0,"prompt_hash":"27fb48b61315f45a8b3d841f5d427abb0862cc10573a986837d80cf7a65cd5e9","question":"[header] How to copy aria montgomery's style without spending hardly anything [title] \" shop \" in your closet. [step] So first, make sure you have the right pieces. If you don't, a substitute is pretty easy to find, such as a denim jacket if you don't have a leather one lying around.\n\nA. If you don't own a denim jacket, who know? Aria will use it and wear it casually over a t-shirt or jean jacket when she enjoys herself. [title] Wear a puppy key piercing if aria has one.\nB. So what are some staples of aria's style? [substeps] Black leather jacket. It's aria's signature, plus it really provides that edge.\nC. It's basically the same except that you have perfect fitted styles, ones worth just about anything. Maybe in yellow, pink, or cream? Chic is something aria really wants.\nD. Then you just have to add accessories like bangles, necklaces, bracelets, or whatever else aria has. Think about transferring the accessories, which will give aria a more realistic look.","prompt":"[header] How to copy aria montgomery's style without spending hardly anything [title] \" shop \" in your closet. [step] So first, make sure you have the right pieces. If you don't, a substitute is pretty easy to find, such as a denim jacket if you don't have a leather one lying around.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7373,"error":null,"output":"B"}
{"question_id":"hellaswag:00008794","item_index":131,"attempt":0,"prompt_hash":"2a6b06131a1f4be419eb907c9c98d42863d2cf83f726fe22b458cebecc9469b1","question":"[header] How to bathe a russian tortoise [title] Get a shallow pan or tray large enough to accommodate your tortoise. [step] Ideally, the container you choose should be made of porcelain or opaque plastic, so that your tortoise will not be tempted to climb out. [substeps] A clean, shallow kitty litter pan or a large porcelain baking dish may be ideal for this purpose.\n\nA. Whatever you choose to use as a tub, only use it for bathing the tortoise. Once it's used to bathe the tortoise, it's no longer suitable for food preparation or other such purposes.\nB. Cold water works even better than lukewarm water, but not as well as hot. [title] Fill a pot or bucket with water.\nC. Be sure that the pan has flushable walls, as well. Most pvc cookers require the lip of the pan to be at least ¼ \" (1/8 cm) below the mod to prevent the reptile from escaping.\nD. [title] Place the tortoise in the pan. [step] The tortoise will then act as his \" warm bath, \" but if you are planning to bathe him yourself, you should place him into it first.","prompt":"[header] How to bathe a russian tortoise [title] Get a shallow pan or tray large enough to accommodate your tortoise. [step] Ideally, the container you choose should be made of porcelain or opaque plastic, so that your tortoise will not be tempted to climb out. [substeps] A clean, shallow kitty litter pan or a large porcelain baking dish may be ideal for this purpose.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7373,"error":null,"output":"A"}
{"question_id":"hellaswag:00002402","item_index":132,"attempt":0,"prompt_hash":"f19b867e9f33bb714e1c53ddcffbcd913cec5ef08b981c11928d2add6582551c","question":"A man wearing protective gear and a helmet is standing and talking to the camera in a paint gun area with a lot of large cushions of different shapes. a man\n\nA. cuts off part of a skin on another man's arm using scissors while sitting and talking.\nB. is siting holding a tool and spraying something on it and adding a small spray we see around the base of his head.\nC. quickly walks in front of the guy talking then reappears in the view and talks to the man.\nD. is holding a pot with large chunks of paint on it in front of him, and wiping it off with a sponge in it.","prompt":"A man wearing protective gear and a helmet is standing and talking to the camera in a paint gun area with a lot of large cushions of different shapes. a man","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7377,"error":null,"output":"C"}
{"question_id":"hellaswag:00008326","item_index":133,"attempt":0,"prompt_hash":"3ffad0882217ea66d0bb3a337f3ef9c76baff2f7cec490d4e7e9e6ee7b5a137a","question":"[header] How to care for your skin over 40 [title] Cleanse your face twice per day. [step] As skin ages, it becomes more sensitive and loses elasticity. Because of this, extra care must be taken with your daily skincare regimen.\n\nA. Wash your face with lukewarm water once in the morning and once in the evening before bed. Use a mild, gentle cleanser with a creamy consistency, which won't strip your skin of its natural moisture.\nB. This includes exfoliating for a few minutes after washing your face. You should also exfoliate once every two days or when you're doing some other activities.\nC. Excessive use of harsh soap or cleansers can lead to inessential aging in your skin. Use the trial and error approach and make sure to remove any potential wrinkles before beginning your skincare routine.\nD. [substeps] Use a great pore cleanser and wash your face. Apply some warm/cold water and a facial cleanser that is ingredients like baby cleanser, and follow it with some facial spray.","prompt":"[header] How to care for your skin over 40 [title] Cleanse your face twice per day. [step] As skin ages, it becomes more sensitive and loses elasticity. Because of this, extra care must be taken with your daily skincare regimen.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7371,"error":null,"output":"A"}
{"question_id":"hellaswag:00004184","item_index":134,"attempt":0,"prompt_hash":"794521593b9f26e96342336029bb0c818c15108244b3bd70b80bcc9270e87448","question":"[header] How to make halloween nail art [title] Select five neon nail polish colors. [step] If possible, go even bolder by painting each nail a different color. [title] Choose black lacquer nail polish so that your skull faces won't chip easily.\n\nA. [title] To start, find a mixer or paint brush and paint it evenly all over the nail. [title] Apply lots of filler once all your color is in place, and let it stand for about a minute so it can dry.\nB. [step] Black lacquer nail polish works best for halloween. [title] Apply neon nail polish to the inner part of each nail.\nC. [title] Fill each nail in with black lacquer paint, to hide any graffiti and algae. [title] Add creepy green eyes and some evil detail by curling your lashes.\nD. [step] Although you will be applying a clear topcoat, preserve your manicure by using strong polish for the details. [title] Apply your neon polish base coat and allow the nail polish to dry.","prompt":"[header] How to make halloween nail art [title] Select five neon nail polish colors. [step] If possible, go even bolder by painting each nail a different color. [title] Choose black lacquer nail polish so that your skull faces won't chip easily.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7371,"error":null,"output":"B"}
{"question_id":"hellaswag:00009927","item_index":135,"attempt":0,"prompt_hash":"229dc3c9861a42ecfcfcb141748c935c9b959dd9ccc0a50c57a32503e4b26255","question":"[header] How to defend against construction contract violation claims [title] Read the complaint. [step] A lawsuit begins with the filing of the \" complaint. \" the person bringing the lawsuit is the \" plaintiff \" and he or she describes the events that led up to the lawsuit.\n\nA. [substeps] Sometimes the person and the plaintiff may file a lawsuit in the same court. In this situation, the court may already have a complaint prepared, even if the people filing the lawsuit do not have one.\nB. The plaintiff also makes a demand for money. As the person being sued, you are the \" defendant.\nC. For example, suppose the person you sued was not following the safety course offered by the construction company and it is called a flood. It states that the contractor paid construction company securities before the flood.\nD. In the complaint, the plaintiff describes the court that has breached the contract provision of the agreement. [substeps] The complaint may be filed in federal court or in small claims court.","prompt":"[header] How to defend against construction contract violation claims [title] Read the complaint. [step] A lawsuit begins with the filing of the \" complaint. \" the person bringing the lawsuit is the \" plaintiff \" and he or she describes the events that led up to the lawsuit.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7370,"error":null,"output":"B"}
{"question_id":"hellaswag:00006911","item_index":136,"attempt":0,"prompt_hash":"7f769d8fee59f9f0f522629ca988d2a01f7e7ea4bf80b8699a857d01716151fa","question":"[header] How to get rid of tobacco smell fast [title] Throw away cigarette butts and ash. [step] Cigarette butts, half-smoked cigars, and ash can leave a strong lingering smell of tobacco indoors. Throwing the butts and ashes into indoor trash bins will only transfer the smell into another room.\n\nA. If you must throw away these items, make sure they go in a trash bag safely. [substeps] If you are using a dumpster for human waste, make sure you throw out all tobacco-related trash you find in the dumpster.\nB. [substeps] As smoking stops, the smells will significantly evaporate from the room. You'll have to account for the temporary smell and let the smoke immediately return.\nC. If the smell still persists, throw away the ones, and stash the equivalent of a lung. [substeps] There are multiple options for accidental combustion when you smoke a cigarette.\nD. Dispose of the butts and the ashes in a bag and move the bag outside. This will eliminate some of the initial smell.","prompt":"[header] How to get rid of tobacco smell fast [title] Throw away cigarette butts and ash. [step] Cigarette butts, half-smoked cigars, and ash can leave a strong lingering smell of tobacco indoors. Throwing the butts and ashes into indoor trash bins will only transfer the smell into another room.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7371,"error":null,"output":"D"}
{"question_id":"hellaswag:00000767","item_index":137,"attempt":0,"prompt_hash":"20e9fa4c27830ede2c4c8933e7186b700f70b763e2708cce11617d196e5e4131","question":"Several clips are shown of people lifting heavy weights as well as jumping on a track. people\n\nA. cheer as they perform quick jumps and their score is shown at the end.\nB. are seen jumping over poles, squatting with weights, as well as performing various other exercises.\nC. run and do jumps while also landing on the mat.\nD. walk up and around the track as well as climbing up a sheer rock face.","prompt":"Several clips are shown of people lifting heavy weights as well as jumping on a track. people","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7373,"error":null,"output":"B"}
{"question_id":"hellaswag:00002317","item_index":138,"attempt":0,"prompt_hash":"38875915063d08ee3a130145a7a67f5170b6c19c82aedb51b47c2e23cdb43e88","question":"A man drinks from a can next to a hole in the snow. he\n\nA. swims slowly down a shallow body of water.\nB. speaks and gestures to the camera.\nC. gets his teeth bitten by a young man.\nD. is drawn in with pick and shovel.","prompt":"A man drinks from a can next to a hole in the snow. he","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7373,"error":null,"output":"B"}
{"question_id":"hellaswag:00008873","item_index":139,"attempt":0,"prompt_hash":"84967b3f0d0b90a64d079d3d9121ea110961caf89151d3944ea053950df05c64","question":"[header] How to be cute [title] Be positive. [step] Cute people are always, and spread their joy for life to others. A cute person's warmth should be able to fill a room and should be felt from a mile away.\n\nA. If you want to improve your positive nature, there are a few things you can do : [substeps] Smile. Do not look like a goof when you smile; just smile like you always do.\nB. Next to adorable, likable, and kind, there are around fifty cute qualities that attract and draw every person to the best of their abilities, from the positive to downright frightening. [substeps] Cheer up people by saying: \" i think you are such a cutie and care about how you look! \" remind people to stop if they are having a terrible day.\nC. [title] Give compliments when you have to be cute. [step] Unfortunately, being cute is essentially a flaw-everyone has such flaws (but not many people like them! ).\nD. If you focus on being positive, your energy will be directed elsewhere and other people will not be able to do the same. [substeps] From a young age, a cute person must be able to talk to and connect with people from all walks of life (from parents to siblings).","prompt":"[header] How to be cute [title] Be positive. [step] Cute people are always, and spread their joy for life to others. A cute person's warmth should be able to fill a room and should be felt from a mile away.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7370,"error":null,"output":"D"}
{"question_id":"hellaswag:00001284","item_index":140,"attempt":0,"prompt_hash":"b02a5b940c27b5089f5fdfc0d1ceef12c7219d87a4bd852805218e9a0241914f","question":"Further in the game the yellow team earns another goal and shakes hands briefly with each other along with clapping their backs in celebratory response. the yellow team\n\nA. loses one goal and he walks away.\nB. scores one final goal at which point the yellow uniformed team members hug each other in response.\nC. continues playing and scoring a goal although they seem to be doing the same thing back and forth.\nD. better mark the match while they're still scoring.","prompt":"Further in the game the yellow team earns another goal and shakes hands briefly with each other along with clapping their backs in celebratory response. the yellow team","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7373,"error":null,"output":"B"}
{"question_id":"hellaswag:00001265","item_index":141,"attempt":0,"prompt_hash":"adffed7abf669a3f810ef2fdea9e29c00fb8c16df5c5edcb70e625042081c1ab","question":"A man in a blue suit dances in the room. The man continues playing the violin. People dance in the room. the man\n\nA. stops playing the violin and the lady in a blue shirt dances the dance properly.\nB. stops playing the violin.\nC. adds a waterfall to the violin.\nD. finishes playing violin and starts talking.","prompt":"A man in a blue suit dances in the room. The man continues playing the violin. People dance in the room. the man","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7372,"error":null,"output":"B"}
{"question_id":"hellaswag:00004579","item_index":142,"attempt":0,"prompt_hash":"e7b0bcfc0f0a22949cabad8d65b226f67828824a5f58b39415a23bddc47ffc25","question":"[header] How to use breast pads [title] Remove adhesive tabs. [step] Some disposable breast pads have sticky pieces on them to hold them in place. The sticky pieces also prevent sliding in your bra.\n\nA. Remove the sticky tabs using a pair of pliers or a windex tool. Slide the pads off and discard them.\nB. If you are using nursing pads with baby-safe adhesive, take off the tabs to expose the sticky bit. Then position your pad just above where you want to apply it.\nC. If the sticky pieces need to be removed, follow the instructions on the label. [substeps] Remove only the tabs that stop your bra from fastening.\nD. This is an option that you can purchase online and can be easily removed by dabbing a cotton ball with the sticky side facing. When removing sticky tabs, firmly push down and lift the padding so that it pulls away from your breast.","prompt":"[header] How to use breast pads [title] Remove adhesive tabs. [step] Some disposable breast pads have sticky pieces on them to hold them in place. The sticky pieces also prevent sliding in your bra.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7373,"error":null,"output":"C"}
{"question_id":"hellaswag:00003604","item_index":143,"attempt":0,"prompt_hash":"6c524746f637d26ad8bd498174240a881797391504eb9a09cdc7f40ed65dc50c","question":"[header] How to deal with a perfectionist spouse [title] Approach their perfectionism pragmatically. [step] Use labeling and reframing as gentle ways to point out your spouse's unnecessary perfectionism. Sometimes, perfectionists make their own lives much more difficult than they need to be and may need their rigidity pointed out in a compassionate way.\n\nA. There's no reason to let criticisms go unmentioned in this article. Instead, wrap things up and lead by example : [substeps] Add a general observational discouragement statements, like \" i do not always know how to make some simple changes in my life.\nB. This response will help you feel that your spouse is noticing your lack of perfectionism. [substeps] For example, your spouse may refuse to do chores or take care of all their housework.\nC. [substeps] Stress should be rooted into perfectionism rather than outright blatant perfectionism. You are just as responsible for these thoughts as your spouse is.\nD. Especially if your spouse is missing out on something fun, point this out to them or make a separate decision on how to handle the situation. [substeps] For example, say, \" we don't have to have everything done before your family arrives.","prompt":"[header] How to deal with a perfectionist spouse [title] Approach their perfectionism pragmatically. [step] Use labeling and reframing as gentle ways to point out your spouse's unnecessary perfectionism. Sometimes, perfectionists make their own lives much more difficult than they need to be and may need their rigidity pointed out in a compassionate way.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7370,"error":null,"output":"D"}
{"question_id":"hellaswag:00002844","item_index":144,"attempt":0,"prompt_hash":"cf1c88d9807ec54274dcb24528ee86c98def0369db6a66b4b3b753265384d202","question":"The girls spins repeatedly and catches her baton. The girl spins and two batons roll in and she grabs each. the girl\n\nA. throws the baton down and spins the baton around her body.\nB. spins again and the other one moves and falls.\nC. spins, catches her baton and poses for her finish as the crowd claps.\nD. forgets the baton spins and lands on her bedroom floor.","prompt":"The girls spins repeatedly and catches her baton. The girl spins and two batons roll in and she grabs each. the girl","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7371,"error":null,"output":"A"}
{"question_id":"hellaswag:00008020","item_index":145,"attempt":0,"prompt_hash":"78020623367667daf50d5e3b3ad819d5fcd77f3d16ae9754b0cd3e22c0ad838f","question":"[header] How to politely tell your parents you don't like your christmas present [title] Open your gift. [step] This seems obvious, but you don't know whether you will like it until you have seen it! [title] Thank them for your present. [step] Your parents have obviously tried hard, so you must be grateful of that.\n\nA. Say something like \" thank you mum and dad, i know you tried hard \" [title] Apologise, and tell them why you don't like it. [step] Saying \" i hate my stupid present \" won't get the best results.\nB. Say \" i really appreciate all of the effort you put into this gift and i want to thank you very much \" and then move on to asking them if you need anything. [title] Tell them how much you appreciate it.\nC. If you know why your parents sent you that gift, allow them to say what you had hoped they would. [title] Just say : [step] \" thank you for the present.\nD. [title] Tell them exactly what you like about it. [step] If you usually love christmas, you may like the gift more, but if you really hate it, it might not really help your case.","prompt":"[header] How to politely tell your parents you don't like your christmas present [title] Open your gift. [step] This seems obvious, but you don't know whether you will like it until you have seen it! [title] Thank them for your present. [step] Your parents have obviously tried hard, so you must be grateful of that.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7370,"error":null,"output":"B"}
{"question_id":"hellaswag:00006476","item_index":146,"attempt":0,"prompt_hash":"93499963fdc5952249c6b2038175d8da2b2b7be53fe17718d1aebf6b974ec08a","question":"[header] How to have great skin [title] Wash your face twice a day. [step] Don't over-wash your face, however. You will dry out your skin, which will cause it to produce extra oils to compensate.\n\nA. Getting rid of oily skin on your face might also help you to eat healthier. [substeps] You can use a cleanser containing salicylic acid, such as lady gotham a, or you can buy cleansers with amino acids and vitamin c.\nB. Instead, wash your face in the morning when you wake up, and in the evening before you go to bed. Use a gentle face wash designed for your skin type.\nC. [substeps] If your skin is greasy or needs to be oily, you can mix 1 to 3 drops of oil into a brush or your fingertips. Apply this oil to your face in an upward direction (above and below your nose) using your fingertips.\nD. Body wash will help hydrate your face, but will also help it to stay moisturized. Apply sunscreen when you go outside, and use an spf 30 sunscreen once a day.","prompt":"[header] How to have great skin [title] Wash your face twice a day. [step] Don't over-wash your face, however. You will dry out your skin, which will cause it to produce extra oils to compensate.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7370,"error":null,"output":"B"}
{"question_id":"hellaswag:00008078","item_index":147,"attempt":0,"prompt_hash":"651b08c501805f20bfb8f52d77a5829cf01fddbcaffba065080628a0a92c0844","question":"[header] How to clean a caique parrot cage [title] Replace the liner at the bottom of the cage. [step] The liner is usually made of newspaper or other paper. Using a face mask and gloves, take out the liner and replace it with fresh paper once a day.\n\nA. [title] Clean the caique's eyes as needed. [step] Caique's eyes are flat and do not change colour very often.\nB. [title] Remove any food feces from the interior of the cage. [step] You need to remove all the other food and feces off of the interior of the cage, along with any droppings, urine, and bacteria.\nC. This will keep the cage free of droppings and stains. [substeps] If you have layered the paper in the cage, you can remove and replace only the top layer as long as water or droppings have not soaked through.\nD. [substeps] As your caique gets older, it will extend its life by 7 , 000 feet (15 , 000 m). Don't spend more than 18 hours each week with your caique bathing.","prompt":"[header] How to clean a caique parrot cage [title] Replace the liner at the bottom of the cage. [step] The liner is usually made of newspaper or other paper. Using a face mask and gloves, take out the liner and replace it with fresh paper once a day.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7371,"error":null,"output":"C"}
{"question_id":"hellaswag:00000744","item_index":148,"attempt":0,"prompt_hash":"f2824f86b4317069a32cd0f57a7bbfaadf05163bb194497d69163dbc468d6f7b","question":"A few people are seen standing around various fooseball tables playing games with one another. the people\n\nA. hold out bowls and pans in to one another and point to one another.\nB. stick hands into cups before pulling them together.\nC. continue to push the ball around back and fourth while the camera focuses on one game.\nD. continue hitting the ball back and fourth to one another while one throws the ball up in the air and runs away.","prompt":"A few people are seen standing around various fooseball tables playing games with one another. the people","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":7370,"error":null,"output":"D"}
{"question_id":"hellaswag:00003470","item_index":149,"attempt":0,"prompt_hash":"29d55dc2a9598dd750b84fdd95a23ec96b8384d79b2eb04f8bde9f8ff7d21e50","question":"[header] How to dress rugged (men ) [title] Shop for pants. [step] Great rugged outfits start with the right pants. Shop for jeans, khakis and canvas with straight-cut legs, and avoid any flashy graphics or loud colors.\n\nA. Also, make sure you try on pants to ensure they fit comfortably. Rugged styles are about utility, so the pants shouldn't be extremely tight or extremely baggy: look for \" relaxed fit \" or \" straight-cut \".\nB. Men should try slacks, dress pants, or street clothes for a rugged look. [substeps] Choose gut-length, flared, or boot-cut trousers.\nC. A good rule of thumb is to dress in the right pair of pants, and balance it with that. Consider clothing tucked into your upper thighs with high-rise, thigh high, or flare pants.\nD. Lightweight and comfortable feet are always a major consideration when shopping for a rugged shirt. [substeps] These hiking boots, while lightweight and appropriate for longer, are a must.","prompt":"[header] How to dress rugged (men ) [title] Shop for pants. [step] Great rugged outfits start with the right pants. Shop for jeans, khakis and canvas with straight-cut legs, and avoid any flashy graphics or loud colors.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":7370,"error":null,"output":"A"}
{"question_id":"hellaswag:00003160","item_index":150,"attempt":0,"prompt_hash":"4d6c34b424b14257668fb6fb43e2cc8cb89b6ba7ea92772621407d1b5e33d6fa","question":"He is attempting to catch a fish through a hole in the ice. He reels in a fish, and removes the hook. he\n\nA. continues reeling in a fish with the camera.\nB. hits the fish with the fishing pole while still reeling it in.\nC. stands up and moves off camera.\nD. then presents the fish and reel in a hole.","prompt":"He is attempting to catch a fish through a hole in the ice. He reels in a fish, and removes the hook. he","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6377,"error":null,"output":"B"}
{"question_id":"hellaswag:00005086","item_index":151,"attempt":0,"prompt_hash":"564f62341ea30e32c9f33946b4161f3a8d7f39d3238e2cc5ff3ec865ef056761","question":"[header] How to grow out natural hair from relaxers [title] Dedicate yourself to your journey of growing out your natural hair from relaxers. [step] Without dedication, you will most likely give up. This journey is not short, but all the while it's worth it.\n\nA. [substeps] Try to buy relaxers that incorporate natural ingredients like crème fraiche and cinnamon. These low-caffeine powders have been shown to improve hair body functions.\nB. For many, getting out your hair is an invaluable tool. For many, it's a long journey, but oftentimes, the remains can be saved for no reason at all.\nC. [title] In transitioning to natural hair, keep your hair moisturized at all times. [step] It's critical that you do so.\nD. [substeps] Cultivate your natural hair by eating a nutritious diet that contains lean protein, healthy fats, and vitamin d. replace your sleep deprived meals with healthy ones.","prompt":"[header] How to grow out natural hair from relaxers [title] Dedicate yourself to your journey of growing out your natural hair from relaxers. [step] Without dedication, you will most likely give up. This journey is not short, but all the while it's worth it.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6377,"error":null,"output":"C"}
{"question_id":"hellaswag:00005173","item_index":152,"attempt":0,"prompt_hash":"a70da1093d74e6b6d4723556c88feee062dbc980c0e6f8753d3ad8ffc4ba2b22","question":"[header] How to become a geography master [title] Look at world maps and atlases at your local library or your school library. [step] Though the study of geography is about more than just maps, it can be helpful to have a visual reference. Head to your local library or your school library and look for world maps in color and as large as possible.\n\nA. Do not forget to pay attention to any nearby ordinances or actual maps. [title] Sit in front of your teacher or observe your teacher.\nB. [substeps] Land stories format maps are most often printed with descriptive themes like divisions, the geographical location of one city, and the names of hundred or more countries. There may even be other types of maps that are written in traditional english.\nC. [substeps] A science book can also be useful to provide a visual reference to yourself and your day to day life. In addition to your textbook, try historical guidebooks to help you determine what the geography world is like for native americans in your area.\nD. You may also be able to find a world atlas, which will likely have smaller images of different regions or areas of the world. [substeps] It may also be helpful to have access to a world globe that has up to date names and locations.","prompt":"[header] How to become a geography master [title] Look at world maps and atlases at your local library or your school library. [step] Though the study of geography is about more than just maps, it can be helpful to have a visual reference. Head to your local library or your school library and look for world maps in color and as large as possible.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6377,"error":null,"output":"D"}
{"question_id":"hellaswag:00007113","item_index":153,"attempt":0,"prompt_hash":"69ec00603da07d3a33b34f2c001e56ed9cf251362b7f22b69953af893e9d6246","question":"[header] How to connect your ps3 to a wireless mobile broadband connection using wi fi [title] Google \" ps3 media server \" [title] Download the latest version from the top link listed. [step] The website should be a sub domain of blogspot. [title] Install the software and unless you know what you are doing just set up all defaults.\n\nA. [title] Start up blogspot, this link will bring you to an unwinachable webpage. [title] While the site is downloading it will also say that it allows syncing of your latest are.\nB. [step] After installing you can still connect that ps3 to one of the wireless broadcasts. [title] Work with the tab below for troubleshooting.\nC. [title] Share the folder (s) n your computer that you want to be able to access with your ps3. [title] Now make sure that your wireless is on in both your router, and your ps3.\nD. [title] After installation click the hook button and your ps3 will be connected with a wi-fi connection. [step] ( do not have any wifi connection this way you will need to connect the headset via ac adapter instead.","prompt":"[header] How to connect your ps3 to a wireless mobile broadband connection using wi fi [title] Google \" ps3 media server \" [title] Download the latest version from the top link listed. [step] The website should be a sub domain of blogspot. [title] Install the software and unless you know what you are doing just set up all defaults.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6378,"error":null,"output":"C"}
{"question_id":"hellaswag:00007412","item_index":154,"attempt":0,"prompt_hash":"d086ec983e56aecb73f7caf8e3e480ffb1b9c0f2541e9ae72ac71ee9838b4c90","question":"[header] How to make fake eyelashes look real [title] Purchase a natural-looking pair of false eyelashes. [step] They're sold at most pharmacies and beauty supply stores, and there are a ton of different brands and styles on the market. If your goal is to make them look natural, you want to avoid the dramatically long, the unnaturally spiky, and the outrageously full ones.\n\nA. Instead, find a pair of fake eyelashes that are about the same size as your natural eyebrow. If possible, purchase different colors, sizes, and styles, especially ones that coordinate slightly with the natural look.\nB. If you want them to look real, bigger is not necessarily better. Choose a pair that look like your dream natural eyelashes.\nC. Instead, use a fake eyelash comb or ponytail holder to make fake lashes look natural and glamorous. [substeps] Fake eyelashes are intended to look very real, and you can even find fake ones at most drugstores.\nD. Find a pair that has a natural shine, and look closely at the tips. [substeps] If you have soft, naturally shiny hair, fake eyelashes aren't going to be as glamorous or realistic.","prompt":"[header] How to make fake eyelashes look real [title] Purchase a natural-looking pair of false eyelashes. [step] They're sold at most pharmacies and beauty supply stores, and there are a ton of different brands and styles on the market. If your goal is to make them look natural, you want to avoid the dramatically long, the unnaturally spiky, and the outrageously full ones.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6377,"error":null,"output":"B"}
{"question_id":"hellaswag:00005780","item_index":155,"attempt":0,"prompt_hash":"373d6fb54212ffd797cbccec733f0dc98e5152e227f3cb8cb0e48f718b741435","question":"[header] How to choose an area rug [title] Measure the room. [step] Area rugs come in all different sizes, from welcome-mat to 15' by 15' (450 by 450 cm). It's important to make sure the rug will fit into the room it graces.\n\nA. Ideally, guests should not spend more than 1 hour looking at the rug. If a rug is part of an entire home, such as a bedroom, it may also get lumpy in a pinch.\nB. However, by the same token, don't skimp on size. Many rooms that would look great with an 8' by 10' (243 cm by 305 cm) rug look strange with a 6' by 9' (90 by 135 cm) rug, for instance.\nC. When determining the rug size, consider factors like : [substeps] How large is the area rug? You should not wrap a rug in a length you can wrap around a living room table. Alternatively, you might consider dividing larger rugs into a double room for rooms that may not effectively be next to each other.\nD. Remove any excess furniture to make room for the rug. [substeps] Check the condition of the rug before deciding how you want it to be displayed.","prompt":"[header] How to choose an area rug [title] Measure the room. [step] Area rugs come in all different sizes, from welcome-mat to 15' by 15' (450 by 450 cm). It's important to make sure the rug will fit into the room it graces.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6094,"error":null,"output":"B"}
{"question_id":"hellaswag:00003205","item_index":156,"attempt":0,"prompt_hash":"b38a3da14757fe5732e6c9714e362fa286a53ca18e49456e537fd6a23398edc5","question":"More shots are shown of the man spinning himself around and throwing an object while others measure his throw. he\n\nA. continues spinning himself around, throwing the object up and down through obstacles and zooms in close on himself.\nB. throws the object one more time and his score is shown.\nC. continues spinning himself around moving his body and flipping himself off the side barely missing the throw.\nD. throws more object into the air and spins it a few more times.","prompt":"More shots are shown of the man spinning himself around and throwing an object while others measure his throw. he","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6095,"error":null,"output":"B"}
{"question_id":"hellaswag:00005793","item_index":157,"attempt":0,"prompt_hash":"531bf5611ba297dd1d6f69ca69eeb83522c6995199b71d06b1e906679ff30f0f","question":"[header] How to make a wallet [title] Mark out your dimensions. [step] Use chalk or a pencil to mark the dimensions of your leather before cutting the pieces out. You will need to mark out a large piece of moose skin for the body or base of the wallet and several smaller pieces of embossed cowhide for change pockets and card pockets.\n\nA. You will also need 4 pieces of black twill or cloth for the front dimensions of your wallet. Six pieces of saddle-leather represents four of the four sides of a football.\nB. The moose skin should be about a foot or two over the elbow mark. You can also mark off some scratches or cracks to draw this out and make it easier to trace around the entire inside of the wallet.\nC. [substeps] The moose skin should be be roughly 11 inches (28 cm) long by 7.5 inches (19 cm) wide. Each card pocket should measure roughly 4 inches (10 cm) long by 2 inches (5 cm) wide.\nD. [title] Draw out your cash pocket and funnel, if you have one. [step] Mark the dimensions so you know what the filling in the pocket will be and make sure to color the inside of the inside of the pocket the appropriately colored color.","prompt":"[header] How to make a wallet [title] Mark out your dimensions. [step] Use chalk or a pencil to mark the dimensions of your leather before cutting the pieces out. You will need to mark out a large piece of moose skin for the body or base of the wallet and several smaller pieces of embossed cowhide for change pockets and card pockets.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6096,"error":null,"output":"C"}
{"question_id":"hellaswag:00002267","item_index":158,"attempt":0,"prompt_hash":"129dbe87e2ec201bc1aa1b59e320657683b7992c3923c5ae11c581111ddc6b97","question":"A man mounts a beam in front of a crowd. The gymnast begins to drop, lift, hold, and spin in circles on the high beams for the crowd. he\n\nA. dismounts, throws his arms in the air, raises his feet high in the air and spins.\nB. dismounts from the beam, shaking hands with other competitors.\nC. does a touch down and gets down and grabs his leg.\nD. dismounts gracefully, raising his arms to the crowd before walking away.","prompt":"A man mounts a beam in front of a crowd. The gymnast begins to drop, lift, hold, and spin in circles on the high beams for the crowd. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6095,"error":null,"output":"D"}
{"question_id":"hellaswag:00002027","item_index":159,"attempt":0,"prompt_hash":"af6f4972fea291dc538586720b7f83209ff10493374a95d7413c7ad9c92690f2","question":"Two women are standing on others' shoulders. They lift another woman into the air, balancing her between them before dropping her to be caught. however\n\nA. , the girls below are knocked to the ground.\nB. , after the women drop her like a rock, she bounces to the top of the floor where she responds by flipping along the air.\nC. , the girls loser flips the one on the side.\nD. , the higher the woman is in the air, the harder the impact of her falling.","prompt":"Two women are standing on others' shoulders. They lift another woman into the air, balancing her between them before dropping her to be caught. however","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6095,"error":null,"output":"D"}
{"question_id":"hellaswag:00009419","item_index":160,"attempt":0,"prompt_hash":"7acd14f6458920ab4b00cfa6ed19c2a7ce56cd3359370eed6c7c329da29142d7","question":"[header] How to wear berry lipstick [title] Decide how dark and opaque you want the lipstick to be. [step] There are a wide variety of berry lipsticks on the market, which run the spectrum from light to dark and highly opaque to sheer. Determine how intense you want your berry lip color to be, so you can choose the appropriate lipstick.\n\nA. [substeps] Dark, opaque berry lipstick can look very harsh if you have fair skin. That doesn't mean you can't wear a dark berry lipstick if you have pale skin.\nB. [title] Choose lipstick you are okay with using. [step] The quality of your lipstick can have a big impact on your hair color.\nC. [title] Lightly fill a funnel with lip gloss. [step] Once you've selected the color of lipstick you want to use, carefully fill a funnel with lip gloss.\nD. Some popular berry lipstick brands include : [substeps] Barbacoa black petroleum jelly [title] Find a one-half inch scoop to stick your lip with. [step] Tip your head back so you can see the object in your hand and your lip.","prompt":"[header] How to wear berry lipstick [title] Decide how dark and opaque you want the lipstick to be. [step] There are a wide variety of berry lipsticks on the market, which run the spectrum from light to dark and highly opaque to sheer. Determine how intense you want your berry lip color to be, so you can choose the appropriate lipstick.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6464,"error":null,"output":"A"}
{"question_id":"hellaswag:00002089","item_index":161,"attempt":0,"prompt_hash":"68b7d3fe6ec225ebac1ba12f9de0e99c4f83ca3dc6282cee6d231cdfab24a1b1","question":"When the gates go down they all go as fast as they can through the obstacle of dirt. There are a few people sitting by in the stands watching as the bikers go off. they\n\nA. arrive to a dirt covered area where there are lots of tents and gear on.\nB. even show some of the kids waiting and having a good time watching the race.\nC. continue going as riders are parked on the trail waiting to ride the bikes.\nD. ride the dust bikes down the track only stopping to rest before leaving.","prompt":"When the gates go down they all go as fast as they can through the obstacle of dirt. There are a few people sitting by in the stands watching as the bikers go off. they","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6470,"error":null,"output":"B"}
{"question_id":"hellaswag:00001627","item_index":162,"attempt":0,"prompt_hash":"ec3f0d5faafc67d1ff0d5ad2815c9d5fc28c408de4f873762c0bdfec108aa2c6","question":"A grinder machine is shown on a patio. A man talks while he shows the parts of the machine. he\n\nA. points to metal parts scattered on the floor.\nB. then shows how to use the grinder to complete a work out as someone watches.\nC. places candy inside of wax, then he tapes it onto the machine.\nD. then demonstrates how the machine is used.","prompt":"A grinder machine is shown on a patio. A man talks while he shows the parts of the machine. he","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6470,"error":null,"output":"D"}
{"question_id":"hellaswag:00009168","item_index":163,"attempt":0,"prompt_hash":"54ac3b2a251375a94b1d217d13fdcf788835bf409ff5151b6cd262f3a36cb3c3","question":"[header] How to make a pig pin brooch [title] Make the pig's head pieces. [step] Draw a circle for the face on a piece of paper. Choose the size for the circles according to what you consider as appropriate for wearing as a brooch.\n\nA. On a sketch, sketch the typical shape of the pig's head (stick-like or oval-like) on cardboard. Use an orange pencil to incorporate the eyes, ears, and mouth.\nB. Colored paper or metallic paper works best, but blue, black, and yellow are good. [substeps] If you want to only use two pin colors, assemble two circles on some construction paper in the color of your choice.\nC. Cut the circle out; this becomes the template for the pig's head. Also make a small circle for the snout and cut this out too.\nD. [substeps] For men, it's better to use a fist size circle for the tailbone. A half circle for each eye will suffice.","prompt":"[header] How to make a pig pin brooch [title] Make the pig's head pieces. [step] Draw a circle for the face on a piece of paper. Choose the size for the circles according to what you consider as appropriate for wearing as a brooch.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6464,"error":null,"output":"C"}
{"question_id":"hellaswag:00009730","item_index":164,"attempt":0,"prompt_hash":"8d2f4aec57dc6ce7d4801c75b1b4f420bfdb3b4826410c39adcd5374f0d8f894","question":"[header] How to get great lips [title] Use a lip scrub to exfoliate your lips. [step] The skin on your lips is delicate, and it easily dries and begins to flake. If you have dry, cracked lips, you need to condition your lips instead of trying to cover them up.\n\nA. You can use a lip scrub made with honey, olay, or vitamin e in place of the regular lip scrub at home, or from the grocery store. [substeps] You don't need to apply it overnight, though.\nB. Get a lip scrub specifically made for lips, or mix up a batch of peanut butter and honey with a few drops of red food coloring and a dash of essential oils. [substeps] Cover your lips with a clean washcloth before applying the lip scrub.\nC. Apply a lip scrub just as you would your face. Stick with the first two to three times daily for best results.\nD. A light lip scrub can do the trick in a matter of minutes. There's no need to buy a special scrub, just mix one up at home.","prompt":"[header] How to get great lips [title] Use a lip scrub to exfoliate your lips. [step] The skin on your lips is delicate, and it easily dries and begins to flake. If you have dry, cracked lips, you need to condition your lips instead of trying to cover them up.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6466,"error":null,"output":"B"}
{"question_id":"hellaswag:00001126","item_index":165,"attempt":0,"prompt_hash":"8e3c157a9d83b88f698225048f9ad7a7a1e439bccc750e8f6ffa5e9fc46f0b87","question":"Arnold schwarzenegger in a gold shirt is playing the violin on stage. The crowd dances sarcastically to his music. he\n\nA. leaves the stage in frustration.\nB. drops another string on his violin.\nC. finishes playing and the audience applauds intensely.\nD. is drunk and wearing an outfit of red knee length pants and black top.","prompt":"Arnold schwarzenegger in a gold shirt is playing the violin on stage. The crowd dances sarcastically to his music. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6468,"error":null,"output":"A"}
{"question_id":"hellaswag:00001901","item_index":166,"attempt":0,"prompt_hash":"aa4f7ea7687a7cf3d2b3229164ebe85e2c8ca5fbbe60dd53383d200f1213c6b1","question":"Men are running in a field then on a city street. we\n\nA. zoom in on the man's finger tips.\nB. see men in rafts in river.\nC. see a man on a large brown horse chase the goalie.\nD. see a lady and a man interviewed and see a store window.","prompt":"Men are running in a field then on a city street. we","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6468,"error":null,"output":"D"}
{"question_id":"hellaswag:00002829","item_index":167,"attempt":0,"prompt_hash":"33965bcc686dbffef71a03f8493238073533a3e32781b75e6e2a33f3eb973d9e","question":"A man is seen holding a sharp knife in his hands while the camera zooms in and the man points to two boxes. He then glides the knife along the boxes and shows off the sharpened knife to the camera. he\n\nA. ends by holding the knife and smiling to the camera.\nB. rubs the knife on a box and continues to sharpen the knife.\nC. then shows the knife to the camera.\nD. points to the boxes more and slowly pulls back the knife and continues to may down the box.","prompt":"A man is seen holding a sharp knife in his hands while the camera zooms in and the man points to two boxes. He then glides the knife along the boxes and shows off the sharpened knife to the camera. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6468,"error":null,"output":"A"}
{"question_id":"hellaswag:00002524","item_index":168,"attempt":0,"prompt_hash":"9608202b47af593d4af3493053a5d2941151d8d2c96e20e319d03c57625d761d","question":"A grroup of men, boys, and girls are playing dodgeball on an indoor court. they\n\nA. run back and forth, playing dodgeball by hitting the nets.\nB. throw a series of balls at one another, trying to dodge and catch them.\nC. run in circles, while one hops many times around the court.\nD. bounce back and forth, trying to scoring.","prompt":"A grroup of men, boys, and girls are playing dodgeball on an indoor court. they","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6467,"error":null,"output":"B"}
{"question_id":"hellaswag:00007327","item_index":169,"attempt":0,"prompt_hash":"bc04eb11e97e78b67d0e04b6c776b6901b0de3beca10313253fd693deeecfcf2","question":"[header] How to be a good step mom [title] Give your stepchildren time. [step] Women are more likely than men to feel responsible for making their stepfamilies perfect. However, remember that most stepchildren will need time to adjust.\n\nA. You may find that you miss being around your friends, jobs, or family members who are not ideal for you or your kids. Serve children as you would grandchildren when they are old enough.\nB. Set aside time for children in your care as children will feel less neglected. [substeps] Break your kids out of home as much as possible.\nC. Some may be open from the start, while others may be more reluctant to engage. [substeps] Recognize that the transition is hard for them.\nD. Do not call, text, and ruin plans with them. Allowing for extra time for your children will make their natural changes much more cathartic than time away from them.","prompt":"[header] How to be a good step mom [title] Give your stepchildren time. [step] Women are more likely than men to feel responsible for making their stepfamilies perfect. However, remember that most stepchildren will need time to adjust.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6463,"error":null,"output":"C"}
{"question_id":"hellaswag:00002552","item_index":170,"attempt":0,"prompt_hash":"eba70467387c457c7ec71d49606f7da5754c3d968cc67b8bfa17febed30c7977","question":"The woman begins to mimic the motion of typing with her fingers and eventually begins to demonstrate sit ups. the woman\n\nA. pushes back the chair and returns to doing sit ups.\nB. pauses to adjust to the position in which she's standing and starts to demonstrate sit ups again.\nC. turns on her side and then lays down on the floor while motioning to her stomach and still talking to the camera.\nD. leans back and begins to do sit ups with her feet.","prompt":"The woman begins to mimic the motion of typing with her fingers and eventually begins to demonstrate sit ups. the woman","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6466,"error":null,"output":"D"}
{"question_id":"hellaswag:00002313","item_index":171,"attempt":0,"prompt_hash":"dbe29adc376a2b9935e9c9a523d132eada3e507e37458ef27907344c794d74ea","question":"First the man steps onto his skateboard and a car rides past. then both of the men\n\nA. push the skateboards back and forth on the dirt path.\nB. take their skateboards and run around an area.\nC. ride the skateboards at the same time in the middle of the streets.\nD. ride their skateboards in a race then take off and ride on the freeway in several curves.","prompt":"First the man steps onto his skateboard and a car rides past. then both of the men","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6465,"error":null,"output":"C"}
{"question_id":"hellaswag:00003358","item_index":172,"attempt":0,"prompt_hash":"b7c07f766adee6277e85fb634cf282b3bc6bf9fc1b1a81ce070ed962e746bc38","question":"[header] How to deal with difficult teenagers [title] Spend quality time together. [step] Connection between the teenager and parent or guardian is a strong protective factor against harm such as emotional distress, risky sexual behaviors, and substance use. [substeps] Schedule regular family activities like eating dinner together or having game nights.\n\nA. Schedule holiday hours, vacations, and social events with the teenager or partner before the teen or guardian takes over. You might ask your teen to babysit a few nights a week for an hour.\nB. If their parents or guardians aren't very close, you can play board games with them or watch a movie together. Some teens who are raised in extremely restrictive environments may develop addictions like alcohol.\nC. You can also get to know the individual person more openly than if you didn't already. Get a hand in all their decisions, especially if that person expressed knowledge of teen problems.\nD. Spend face-to-face time on a weekly basis. Take your teen out to lunch or dinner without the rest of the family.","prompt":"[header] How to deal with difficult teenagers [title] Spend quality time together. [step] Connection between the teenager and parent or guardian is a strong protective factor against harm such as emotional distress, risky sexual behaviors, and substance use. [substeps] Schedule regular family activities like eating dinner together or having game nights.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6461,"error":null,"output":"D"}
{"question_id":"hellaswag:00002882","item_index":173,"attempt":0,"prompt_hash":"4d6601d9813f09ed0caa95d440dc43c3e27c374af0ad101f014b85d38098f64c","question":"The scene changes back to the boy. The camera zooms in on the controller. The scene goes back to the game. there\n\nA. is a close up of the boy's face.\nB. is still a player on the street playing the game.\nC. is another closing image on the screen.\nD. are several pool balls in play.","prompt":"The scene changes back to the boy. The camera zooms in on the controller. The scene goes back to the game. there","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6465,"error":null,"output":"A"}
{"question_id":"hellaswag:00009306","item_index":174,"attempt":0,"prompt_hash":"d25a5e6b19a5aba3df35a2ff1f6fdb6a7d906e27c83b9b43957380c86c6bc1f7","question":"[header] How to do indian meditation [title] Sit on the floor or in a chair. [step] Find a place where you can sit comfortably for at least ten minutes. It could be indoors or outside.\n\nA. You could even wear the same socks or wear the same footrest. Although there are many practicing yoga, it is not truly necessary to practice it.\nB. Don't meditate in a bad home, as, your home will be home to a chronic and pushy person, and it will likely get tired again. [substeps] If you can, find a warm place that will allow you to sit comfortably for at least 30 minutes, preferably in a room that is not hot or stuffy.\nC. [substeps] You could also hold a prayer or light candle. If you can't find a comfortable place in your home, try reading a magazine or doing a search for guided meditation websites.\nD. What matters most, however, is that it is quiet and free from distracting noise like music, television, or people talking. [substeps] Not all noise is bad in the vipassana tradition.","prompt":"[header] How to do indian meditation [title] Sit on the floor or in a chair. [step] Find a place where you can sit comfortably for at least ten minutes. It could be indoors or outside.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6462,"error":null,"output":"D"}
{"question_id":"hellaswag:00006474","item_index":175,"attempt":0,"prompt_hash":"2aff841804948ec4fea239a4a73846c363162920e7a730cf46549eb496815e92","question":"[header] How to breed budgies [title] Decide whether you really want to breed budgies. [step] Breeding budgies takes time and isn't easy. You need to cash out on supplies and feed, check on the budgies often, clean the cage daily and set aside extra time for caring for the chicks.\n\nA. You'll need to find a reputable breeder to market the farm and breed budgies for you. [substeps] Since budgies are never born of a mother, the only people that can breed yours are the chicks themselves.\nB. [title] Find a suitable room to breed budgies. [step] The room should be quiet, protected from predators including pets, have no disturbances and the only person entering the room should be you.\nC. Fertile chicks are ideal breeders, since they can mature when they are not nursing. It's also a good idea to hire babysitters.\nD. [substeps] Budgies develop quickly and i don't want to wait until they grow bigger to breed them as it's easier for embryos to grow. [title] Set up habitat for your budgies.","prompt":"[header] How to breed budgies [title] Decide whether you really want to breed budgies. [step] Breeding budgies takes time and isn't easy. You need to cash out on supplies and feed, check on the budgies often, clean the cage daily and set aside extra time for caring for the chicks.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6461,"error":null,"output":"B"}
{"question_id":"hellaswag:00009759","item_index":176,"attempt":0,"prompt_hash":"414352888773b2c7c83b4d94aff23355bd166aae0976881029f7d7d068cccdd3","question":"[header] How to decorate school supplies [title] Use stickers. [step] One of the easiest and best ways of decorating your notebooks, binders, and other school supplies is to use stickers. Head to the craft store or the hobby store and find some stickers that you like, and decorate your notebooks.\n\nA. Consider pressing and gluing stickers for extra fun. [substeps] Glue art or designs onto your notebooks and binders in colorful boxes.\nB. [substeps] Apply the stickers in unusual patterns for a neat effect. Even if you've just got a bunch of football stickers, if you make them into the shape of a face, or a peace sign, it might be cool.\nC. Brush (or paste) your stickers onto the outside of your notebooks. [substeps] You could also use a tacky glue on the inside or edges of the corner of the tabs of your notebooks as well.\nD. They can be as small as math stars, as large as bike wheels, or as big as the words in english. [substeps] You can often find stickers online in large baskets and inside themed packs of things.","prompt":"[header] How to decorate school supplies [title] Use stickers. [step] One of the easiest and best ways of decorating your notebooks, binders, and other school supplies is to use stickers. Head to the craft store or the hobby store and find some stickers that you like, and decorate your notebooks.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6465,"error":null,"output":"B"}
{"question_id":"hellaswag:00008822","item_index":177,"attempt":0,"prompt_hash":"6b604237a09bb7c3b9e6cc004f919d63c144edcff5cfc2e670b8b4527e16ef29","question":"[header] How to have a cool facebook profile [title] Go easy on the apps. [step] Facebook is a site to keep in touch with your friends, not to fool around with farmville all day. They may be tempting and addicting, but don't add too many, because they will be just a nuisance when it comes to notifications.\n\nA. These apps don't quite bring you together, and remain a nuisance for too long. Here are some other useful dos and don'ts for facebook : [substeps] Facebook is not like a website; rather, it is a google post page.\nB. [substeps] Try to add only friends you really care about. Most facebook applications don't like broken messages.\nC. [title] Watch your status-try not to put everything you do in there. [step] If there is something you want your friends to hear or something cool happened, put it there.\nD. It's a fun and easy way to make new friends though! [title] Sync your facebook page to your facebook and twitter accounts. [step] There's too much trash! [title] Get more positive about your being cool and having a cool profile.","prompt":"[header] How to have a cool facebook profile [title] Go easy on the apps. [step] Facebook is a site to keep in touch with your friends, not to fool around with farmville all day. They may be tempting and addicting, but don't add too many, because they will be just a nuisance when it comes to notifications.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6464,"error":null,"output":"C"}
{"question_id":"hellaswag:00009856","item_index":178,"attempt":0,"prompt_hash":"d45497c8605cd751332e73095aec6f0c7c6c372057bc5c68de28a3ab8ffc3ff3","question":"[header] How to be a good au pair [title] Gain babysitting experience. [step] If you want to be an au pair, parents have to be able to trust you with the well-being of their children. You can achieve this by ensuring that you have plenty of experience with babysitting or caring for children of various ages.\n\nA. This can vary from young children to at-home caretakers at various ages. Consider looking into becoming an au pair who shows some of the skills required to run a household.\nB. [substeps] Most au pairs want experience in caring for their children. You can gain babyhood teaching experience, social skills, or even an apprenticeship with a relative if you have an interest in that.\nC. Try caring for family members, neighbors' children, or get a job at a local daycare center. The more experience you have with children, the more comfortable parents will be with you.\nD. [substeps] If you love to cook, babysitting uses that ability to please your parents. If you are a good cook, babysitting is a combination of cooking and caring for children.","prompt":"[header] How to be a good au pair [title] Gain babysitting experience. [step] If you want to be an au pair, parents have to be able to trust you with the well-being of their children. You can achieve this by ensuring that you have plenty of experience with babysitting or caring for children of various ages.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6462,"error":null,"output":"C"}
{"question_id":"hellaswag:00005675","item_index":179,"attempt":0,"prompt_hash":"6824e075a92454b1b5b573ec16f791cb9fe6679376c0a99f5357cbb532d38b2a","question":"[header] How to texture walls [title] Lay painter's plastic around the floor of the room. [step] Place painter's plastic on windows, affixing them with blue tape to hold them in place. Cover any places that you do not want to get dirty.\n\nA. [title] Put some fiberglass sheeting around the room. [step] A piece of fiberglass could potentially make the wall mortar soften in spots.\nB. Painter's plastic isn't inexpensive; you can get it at home. The same is true for the bathroom too.\nC. Retexturing walls is a messy process, so make sure all the painter's plastic sheets overlap. [title] Apply a 1/8 inch (0.\nD. A painter's plastic sheet works well when side-swept or that has been washed recently. [title] Use \" liquid \" roller coats as ventilators on ceiling or wall surfaces.","prompt":"[header] How to texture walls [title] Lay painter's plastic around the floor of the room. [step] Place painter's plastic on windows, affixing them with blue tape to hold them in place. Cover any places that you do not want to get dirty.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6462,"error":null,"output":"C"}
{"question_id":"hellaswag:00003569","item_index":180,"attempt":0,"prompt_hash":"64fe65957e3d435d75de791369734d600c4dc6c086e789aa2c9079343803c2ab","question":"[header] How to avoid parent teacher conflicts [title] Get acquainted in your child's presence. [step] Many schools host a \" meet the teacher \" event at the start of a new school year to let parents and students introduce themselves to teachers. Attending this event sets the tone for positive interaction with your child's teacher.\n\nA. It also shows your concern for your child's educational experience. [substeps] These events can be busy and aren't the best environment for extended dialogue.\nB. The meet and greet event will be a fun day for both parents and students. [substeps] The meet and greet event occurs every tuesday morning at 9:00 am eastern time.\nC. [substeps] If you are hosting the meet-and-greet event, be in the same room with your child and seek his/her consent to meet the teacher in advance. You may also want to consider including other parents or fellow students.\nD. [substeps] Take note of the features your child must have that matter to you. Many preschools have offered free \" training days \" for those who plan on working in a specialized field.","prompt":"[header] How to avoid parent teacher conflicts [title] Get acquainted in your child's presence. [step] Many schools host a \" meet the teacher \" event at the start of a new school year to let parents and students introduce themselves to teachers. Attending this event sets the tone for positive interaction with your child's teacher.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6464,"error":null,"output":"A"}
{"question_id":"hellaswag:00001264","item_index":181,"attempt":0,"prompt_hash":"fb8d47fe74f0c883f433b152c267c221145cfed5422f96b213a7a6eaa407fea8","question":"A woman in a green dress smokes and holds her ears. A man in a blue suit dances in the room. The man continues playing the violin. people\n\nA. walk around the beach.\nB. pass him an the violin and the music ends.\nC. dance in the room.\nD. in bowling shirts grab bowling pins and hold them in the air.","prompt":"A woman in a green dress smokes and holds her ears. A man in a blue suit dances in the room. The man continues playing the violin. people","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6464,"error":null,"output":"C"}
{"question_id":"hellaswag:00007046","item_index":182,"attempt":0,"prompt_hash":"1b2b750650eb2306be23a33efaec313f23662585462adaef016bb1e8fab93913","question":"[header] How to make a duct tape iphone case [title] Measure your phone, then add 1 inch (2.5 centimeters) to each side. [step] Use a ruler to measure the length and width of your phone. Add 1 inch (2.5 centimeters) to each measurement.\n\nA. Label and record the measurements as \" 1/2 inch. \" [substeps] A sliding scale is also helpful.\nB. [title] Make two duct tape sheets based on those measurements. [step] Cut two strips of duct tape based on your length measurement.\nC. The thinner the duct tape, the larger your case will get. [substeps] If you used superglue and are using some cheap thin cardboard tubes instead, you can skip this step and simply measure a bit.\nD. [title] Cut out rectangular tiles onto a piece of cardboard. [step] You can cut them just round, or you can cut them in a bunch to form a rectangular box.","prompt":"[header] How to make a duct tape iphone case [title] Measure your phone, then add 1 inch (2.5 centimeters) to each side. [step] Use a ruler to measure the length and width of your phone. Add 1 inch (2.5 centimeters) to each measurement.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6508,"error":null,"output":"B"}
{"question_id":"hellaswag:00004616","item_index":183,"attempt":0,"prompt_hash":"99284f5a2a499acddbc4ca06ecd444d900898280ad95e7680d8b6f00fb23c519","question":"[header] How to feel pretty [title] Understand your own prettiness. [step] This is the most important step for feeling pretty. You have to understand that your beauty comes from, not from any outside source.\n\nA. But you have to practice feeling this way. [substeps] Write a list of all the good things about yourself.\nB. It is much easier to find beauty when you believe you deserve it ,. You make the mistake of assuming that people will like your appearance, you assume that part of beauty is just your looks.\nC. It can't be tied back to an inner drive. You have to understand your own beauty and how it works.\nD. Don't abandon your beauty, because you use it. Don't tempt yourself to go flaunt it-looking good is beauty and so is wearing it.","prompt":"[header] How to feel pretty [title] Understand your own prettiness. [step] This is the most important step for feeling pretty. You have to understand that your beauty comes from, not from any outside source.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6508,"error":null,"output":"A"}
{"question_id":"hellaswag:00009160","item_index":184,"attempt":0,"prompt_hash":"126ec72f6057967868e1ce8c79277c42693f3ef641ffc1422c933b90e8811cad","question":"[header] How to dress your plus size body type [title] Dress for a pear shaped figure : [step] The pear is the most common female body type. You can call yourself a pear if your upper body is noticeably smaller than your lower body. If you were to gain five pounds, four of them would go to your hips and thighs.\n\nA. Consult a personal trainer to get an idea of your body type, as pear sizes tend to be smaller in musculoskeletal structures. [substeps] The shape of your body complies with a special relationship with your height.\nB. You may have a long or slender neck, sloping or narrow shoulders, narrow back, small to medium bust, defined waist or full calves. You definitely have very curvy hips, thighs and seat.\nC. However, if you walk or move up high, one of those pounds would go to your waist and the other would go to your hips. For instance, if you are tall but you stand short, you could now look like a pear.\nD. Cut your bikini bottoms in half to accommodate your pear-shape. [substeps] You should also wear a crop top of whatever measurement you're comfortable with.","prompt":"[header] How to dress your plus size body type [title] Dress for a pear shaped figure : [step] The pear is the most common female body type. You can call yourself a pear if your upper body is noticeably smaller than your lower body. If you were to gain five pounds, four of them would go to your hips and thighs.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6507,"error":null,"output":"B"}
{"question_id":"hellaswag:00009899","item_index":185,"attempt":0,"prompt_hash":"d0294bc497d57dbeb72ac36e88079a042d606331a00e60d68cf7412f0cd26597","question":"[header] How to put your hair up with a pencil [title] Find a pencil. [step] It has to be long enough to comfortably fit in your hair. You can, though, use anything that's handy (and stick shaped) like a chopstick, a fork, a toothbrush... [title] Gather your hair into a tight ponytail with your hands.\n\nA. [step] Secure the ponytail with a plastic hair tie or twist as you would with regular hair. [substeps] You may also want to use an elastic hair tie to add some security to your ponytail.\nB. [step] You won't need a rubber band. [title] Hold your hair with your left hand, and with the right, grab your pencil with the sharp side pointing down.\nC. [step] One of the good things about a pencil is that it is easy to keep in place. The smaller it is, the more secure you'll be.\nD. [step] The longer you let go of your hair the shorter and better it'll look. Put a rubber band around it to make it protect your hands from getting tangled in it.","prompt":"[header] How to put your hair up with a pencil [title] Find a pencil. [step] It has to be long enough to comfortably fit in your hair. You can, though, use anything that's handy (and stick shaped) like a chopstick, a fork, a toothbrush... [title] Gather your hair into a tight ponytail with your hands.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6508,"error":null,"output":"B"}
{"question_id":"hellaswag:00002186","item_index":186,"attempt":0,"prompt_hash":"c8548987d81f00620a16de3c6691e33016914fc83041942ebf103a5123695a18","question":"One of the men throws a beer pong ball across the table, into a cup. the other man with him\n\nA. throws a ball across the table, too.\nB. throws another beer ball, which he catches and grabs.\nC. loses and takes off running.\nD. takes another drink.","prompt":"One of the men throws a beer pong ball across the table, into a cup. the other man with him","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6506,"error":null,"output":"A"}
{"question_id":"hellaswag:00008754","item_index":187,"attempt":0,"prompt_hash":"d05d608a4a490f22d9210a6ff8a0e54d8df21bf7281e42af9e4aad81d5013419","question":"[header] How to teach grammar [title] Know who your students are. [step] In order to teach effectively, you need to know who your students are. You should also know what they want or need to learn from you.\n\nA. You should try to guide them in how they interact with you in order to help your students understand you better. [substeps] When taking your students for a class discussion, try to talk to them in a simple way.\nB. This will affect your teaching approach, as well as what you actually teach. [substeps] For example, if you are teaching elementary school kids, you might focus on basic grammar rules that are easy to grasp.\nC. [substeps] Students are usually more interested in vocabulary than in grammar. However, once you tell them a story, they will want to learn to read it so they are better at understanding the material.\nD. [substeps] Be aware that not all students will be able to teach grammar effectively. Some students may struggle with understanding grammar properly; others may be struggling at not continuing because they haven't learned basic grammar rules.","prompt":"[header] How to teach grammar [title] Know who your students are. [step] In order to teach effectively, you need to know who your students are. You should also know what they want or need to learn from you.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6907,"error":null,"output":"B"}
{"question_id":"hellaswag:00000547","item_index":188,"attempt":0,"prompt_hash":"f50b0ea4adc795fef68a38b4ada90a81ea8db4e72c792abb306fb8d5a9cab77b","question":"A man sits on a sail boat in the river. Another sailboat is shown being driven. several people\n\nA. are in the water, trying to make floats.\nB. are interviewed on the docks.\nC. walk beside the boats, guiding them with their oars.\nD. are then seen at various rafts rafting.","prompt":"A man sits on a sail boat in the river. Another sailboat is shown being driven. several people","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6906,"error":null,"output":"B"}
{"question_id":"hellaswag:00006237","item_index":189,"attempt":0,"prompt_hash":"ce016c10fabbbdc1ab626038b207e90d13bb1494fb7932a291e8afc268f9ae13","question":"[header] How to keep children from waking up too early [title] Identify possible reasons for your toddler's early wake time. [step] Many children wake up early in the morning due to noise, hunger, or light. Try to keep a record of your child's sleeping habits for one to two weeks.\n\nA. This will help you provide a good, continuous record of his up-to-the-minute habits. [substeps] You might record breakfast, lunch, and snack times.\nB. [substeps] For instance, if your child wakes up about an hour before naptime, have them break out his ready-made lunch so they can have enough time to eat before naptime. If your child usually wakes up hours before naptime, try a more gradual recording of their wake time.\nC. Note what time your child goes to bed, what time they wake up, and any naps they take during the day. You should try to identify any patterns in your child's sleep, and if noise, hunger, light or too many naps during the day are causing the early wake time.\nD. [substeps] Look on your child's waking times and things they engage in during the day and other times they wake up. You might note bedtimes, work/school times, break-ins, or other times that your child felt awake immediately.","prompt":"[header] How to keep children from waking up too early [title] Identify possible reasons for your toddler's early wake time. [step] Many children wake up early in the morning due to noise, hunger, or light. Try to keep a record of your child's sleeping habits for one to two weeks.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6907,"error":null,"output":"C"}
{"question_id":"hellaswag:00006758","item_index":190,"attempt":0,"prompt_hash":"6df64caa5c0383eae18fb898b4fda72642f3d8c654c2c0e6b2db6f91e120233f","question":"[header] How to make crispy chicken with creamy italian sauce and bowtie pasta [title] Crush the cereal in a food processor until it is reduced to crumbs. [title] Transfer the crumbs to a bowl. [title] Add the salt and flour to the crumbs.\n\nA. [title] Break the chicken into the beaten egg. [title] Sprinkle the grated parmesan over the egg mixture.\nB. [title] Stir to combine the salt and flour into the crumbs. [title] Line up 3 bread loaf pans or pie tins to put the coatings in.\nC. [title] Scoop the salads according to your preferences. [step] Combine the lettuce, mayonnaise, dill, pickled tomatoes, chopped fresh basil, pancetta soup, bell pepper, ramen noodles, gravy and bread crumbs in a large glass bowl.\nD. [title] Mix all of the ingredients until well-combined. [title] Transfer the sauce mixture to a plate.","prompt":"[header] How to make crispy chicken with creamy italian sauce and bowtie pasta [title] Crush the cereal in a food processor until it is reduced to crumbs. [title] Transfer the crumbs to a bowl. [title] Add the salt and flour to the crumbs.","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6906,"error":null,"output":"D"}
{"question_id":"hellaswag:00004316","item_index":191,"attempt":0,"prompt_hash":"25e77f398a2a49513c7e2a125469408b3805a0e5885d923d533dfd0e63ac5f79","question":"[header] How to write movie, music, and video game reviews [title] Watch, play, or listen repeatedly. [step] The first step in any review is to become as familiar as you can with the media product you are reviewing. Spend some time watching, listening, or playing.\n\nA. Research shows how much of the last 40 million-year movie reviews were gained through immersion in the internet. [title] Explore the various audio sourcebooks and forums related to video game reviews.\nB. Always understand what the product is about and get familiar with it. Read the reviews for any inaccuracies that you find.\nC. [substeps] For movies, you should plan on watching the film at least two or three times. The first time, just watch and enjoy as your normally would.\nD. Try not to lean too heavily on that experience. Try to stay inconstant (or at least not get distracted).","prompt":"[header] How to write movie, music, and video game reviews [title] Watch, play, or listen repeatedly. [step] The first step in any review is to become as familiar as you can with the media product you are reviewing. Spend some time watching, listening, or playing.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6906,"error":null,"output":"C"}
{"question_id":"hellaswag:00006623","item_index":192,"attempt":0,"prompt_hash":"5e97f8b85ac318067af161e0ee45adda6589372e5638406eed4b78418302c697","question":"[header] How to make buffalo chicken dip [title] Preheat your oven to 350 degrees. [step] You'll want to cook the entire dip in the oven once assembled to melt the cheese on top and ensure the flavors blend together. That said, the chicken needs to be pre-cooked, so this time in the oven is just to get everything hot.\n\nA. [title] Warm the 8-oz package cream cheese up to room temperature. [step] This simply makes it easier to blend into the other ingredients, preventing chunks of cream cheese in some parts of the dip and none in the others.\nB. [substeps] Make sure that the oven is at temperature by flipping the oven dial and turning it on to 350ºf (180ºc). When the oven reaches 350 degrees f (178ºc), turn off the oven.\nC. [substeps] Alternatively, pre-cooked chicken dip can go the other route and be a little tougher by sticking to a lower oven temperature. Keep in mind that buffalo is a complex, high-calorie sauce.\nD. [title] Pour the olive oil into a deep-fryer or pan. [step] To ensure that the oil cooks evenly, you'll need to make sure the oil is evenly distributed in the bottom of the pan as well.","prompt":"[header] How to make buffalo chicken dip [title] Preheat your oven to 350 degrees. [step] You'll want to cook the entire dip in the oven once assembled to melt the cheese on top and ensure the flavors blend together. That said, the chicken needs to be pre-cooked, so this time in the oven is just to get everything hot.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6375,"error":null,"output":"D"}
{"question_id":"hellaswag:00003305","item_index":193,"attempt":0,"prompt_hash":"1cb90e2854590371501b2654ef26aa7214c98904a7aa11968d8bc984f49bf7ba","question":"[header] How to feed a colicky baby [title] Bathe your baby. [step] Your baby might relax if given a warm bath. To bathe your little one, fill a tub up with a few inches of warm water.\n\nA. As the baby slowly soaks in the water, try to cover their head with a towel. Allow the water to get to your baby's face.\nB. Sit your child in the tub and use a cup to pour warm water over their legs and shoulders. You could also use a mild soap in the water, but be careful not to get any in your baby's eyes.\nC. Then add a fresh section of cling wrap and your baby will gently submerge himself or herself. Keep this in place for 3-4 hours, at which time the baby will sit on your legs and press his or her head against your leg.\nD. Dip a soft cotton towel or cloth in the tub and place it on your child's abdomen. Make sure to get water from its stomach up, as babies' skin is very delicate.","prompt":"[header] How to feed a colicky baby [title] Bathe your baby. [step] Your baby might relax if given a warm bath. To bathe your little one, fill a tub up with a few inches of warm water.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6373,"error":null,"output":"B"}
{"question_id":"hellaswag:00001173","item_index":194,"attempt":0,"prompt_hash":"c3905e5d45c9cdada8de7921558353d56a1bdb8d54c3e4fc9f85a246f66c2b79","question":"There's a lot of children and adults in a large indoor gymnasium with a wood floors and some are rope jumping while others are watching while sitting or standing. a brown skinned girl in the forefront\n\nA. isn't wearing a shirt as she looks around and watches another class but continues to watch the other children gymnasts.\nB. focuses on the rope climbing instructions on her phone.\nC. begins to jump rope very quickly.\nD. of the group leaves the gym and is followed by another child jump rope.","prompt":"There's a lot of children and adults in a large indoor gymnasium with a wood floors and some are rope jumping while others are watching while sitting or standing. a brown skinned girl in the forefront","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6380,"error":null,"output":"C"}
{"question_id":"hellaswag:00007560","item_index":195,"attempt":0,"prompt_hash":"317efb03391cd8f71e9c0fc1bea4f4378a405c4f3b0d5e81d9052cc9e957e485","question":"[header] How to create a picture memorial [title] Locate several of your favorite photographs of this person across a span of time. [step] If they were in your kindergarten class, graduated high school with you and were your best friend well into your 80s, you should have plenty of choices. If you've only known the person for a short time, ask friends and family members for contributions.\n\nA. [substeps] With your photos, consider monopolizing a particular set of photographs of this person. This, in turn, gives the photos a sense of closeness.\nB. [title] Embed some of your favorite art and music elements with the photos. [step] If you can use excel = tabs [title] Find some flashy, fun tags to line the photo with.\nC. Camera roll considerations include arranging various props, cameras or side and lens decisions. [substeps] Arranging materials ahead of time is perhaps the most important determining how to arrange your photographs.\nD. If you do not want to use the original photo, copy the pictures at the neatest quality you want. [title] On a large, clean, dry surface lay out your pictures and try to narrow down your initial selections.","prompt":"[header] How to create a picture memorial [title] Locate several of your favorite photographs of this person across a span of time. [step] If they were in your kindergarten class, graduated high school with you and were your best friend well into your 80s, you should have plenty of choices. If you've only known the person for a short time, ask friends and family members for contributions.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6372,"error":null,"output":"D"}
{"question_id":"hellaswag:00001878","item_index":196,"attempt":0,"prompt_hash":"b0941a928f2c9dc5df93a2e8a8fffcd47404a4118a60e797866543d424cc87e3","question":"Several basketballs players practice basketball both with a coach and alone in an indoor gymnasium. several players\n\nA. dribble a ball and compete against each other during an outdoor basketball game.\nB. practice basketball by dribbling balls and practicing free throws in a interior gym.\nC. enter the gym after the teacher is finished talking.\nD. serve the ball and go into a goal match against raccoons against the wall behind the play rink.","prompt":"Several basketballs players practice basketball both with a coach and alone in an indoor gymnasium. several players","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6378,"error":null,"output":"A"}
{"question_id":"hellaswag:00004736","item_index":197,"attempt":0,"prompt_hash":"092b1244515da943a7bce46a8317c1f1c96f35d077e9e7b79eb92e66fed667af","question":"[header] How to not be afraid the first time you use a tampon [title] Learn about tampons and alternatives. [step] You don't have to use a tampon while on your period. In fact, many people prefer to use pads or menstrual cups.\n\nA. Unless you're on your period at the time you're having your period, there is no one way to get pregnant. In addition, you're equally unlikely to get pregnant on your period.\nB. Tampons provide more freedom in motion and are superior when playing sports, especially those involving water. However, tampons can require some effort with handling or insertion.\nC. Most women find it enjoyable to use a tampon while over the phone or in the shower. A good way to reduce your anxiety is to buy a tampon which has both a clip-on holder and the area where the tampon will be inserted into.\nD. If you're in the flush or not this method will not work for you. Instead, you can either gently use a tampon tip to get an accurate reading, or, if you just prefer to go the old fashioned route, use a tampon refill.","prompt":"[header] How to not be afraid the first time you use a tampon [title] Learn about tampons and alternatives. [step] You don't have to use a tampon while on your period. In fact, many people prefer to use pads or menstrual cups.","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6371,"error":null,"output":"A"}
{"question_id":"hellaswag:00000370","item_index":198,"attempt":0,"prompt_hash":"3ccd713eb76da97c5fdc628b389da69db059151af3cfc56997567195a181d7f4","question":"The guy seems to be practicing how high he can jump by jumping over the pole many times, each time a person raises the pole higher and higher to see if the guy can jump over it. one jump the guy\n\nA. does but the pole falls then he jumps but he fails to land very far.\nB. ensure and then jumps over the pole and does somersaults once he's standing directly over it.\nC. knocked the pole down when he tried to hump over it.\nD. jumps from and lands parallel with the pole pole.","prompt":"The guy seems to be practicing how high he can jump by jumping over the pole many times, each time a person raises the pole higher and higher to see if the guy can jump over it. one jump the guy","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6379,"error":null,"output":"C"}
{"question_id":"hellaswag:00000970","item_index":199,"attempt":0,"prompt_hash":"938d80919abf81057f4022ed753b1279474a7433c9c33ef62e08dddcf2c6aa9a","question":"An elderly woman lays on down against a brown pillow. She holds a harmonica to her mouth and plays it. The light in the room brightens. she\n\nA. continues playing the harmonica and ends by laying down, laying in the bed with her eyes closed and resting her head on a pillow.\nB. finishes playing and laughs.\nC. begins to play the harmonica more.\nD. continues playing and exhales a lot of smoke while turning the harmonica off.","prompt":"An elderly woman lays on down against a brown pillow. She holds a harmonica to her mouth and plays it. The light in the room brightens. she","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6378,"error":null,"output":"C"}
{"question_id":"hellaswag:00007205","item_index":200,"attempt":0,"prompt_hash":"c0ca2646f95dad4067d5d9e4885a8d3826a1d6a1dc9509528b2e6a0415b41201","question":"[header] How to know if your child needs therapy [title] Look for signs or threats of self-harm. [step] Cutting or other acts of self-harm (i.e. burning, scratching, picking at skin, etc. ) are red flags that your child needs professional help.\n\nA. However, even if your child hasn't actually hurt themself, stay alert for threats of them harming themself. This may be clearly spelled out, such as \" i want to die \" or more subtle like, \" i need to punish myself.\nB. The first step to getting help is to see if any of these actions are threatening to harm your child and monitor your child's behavior. [substeps] Warning signs of self-harm include: disrupting social and spiritual activities, or: becoming highly irritable.\nC. [substeps] If your child refuses help or doesn't seem in immediate danger, move the child to a separate room if they refuse help. Avoid wearing gloves when handling your child's body.\nD. However, there are many possible warning signs that your child needs treatment. These are : [substeps] Persistent location: kids might wake up and cut their own clothing and other household items in an attempt to hide their nudity (e.g.","prompt":"[header] How to know if your child needs therapy [title] Look for signs or threats of self-harm. [step] Cutting or other acts of self-harm (i.e. burning, scratching, picking at skin, etc. ) are red flags that your child needs professional help.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6374,"error":null,"output":"A"}
{"question_id":"hellaswag:00003613","item_index":201,"attempt":0,"prompt_hash":"78eae2091422abe79adb6b19c6c4780c601fea2b057f13ffe4a43e71dcc6daed","question":"[header] How to avoid raising a bully [title] Talk to your child. [step] Be interested in your child's life, and check in with them on a regular basis. Ask questions like what they're working on in school, how their friends are, and what's been getting under their skin lately.\n\nA. Ask about their health, and address concerns you may have about what type of bullying they might be dealing with. [title] Watch for signs of the bully's behavior.\nB. [substeps] Give your child your full attention when you talk with them. Kids know when you're tuning them out or just pretending to be interested.\nC. You'll also make sure to inquire about their interests, and try to bring up the things they did well in school. [substeps] It might be helpful for you to ask them questions at school about sports, if anything, while you don't want to come across as a bully.\nD. Basically, make a list of all the events, clothes they've worn, or other activities they enjoy the most. [substeps] Always be observant, and take up as much time as possible to learn about your child's life.","prompt":"[header] How to avoid raising a bully [title] Talk to your child. [step] Be interested in your child's life, and check in with them on a regular basis. Ask questions like what they're working on in school, how their friends are, and what's been getting under their skin lately.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6371,"error":null,"output":"B"}
{"question_id":"hellaswag:00005359","item_index":202,"attempt":0,"prompt_hash":"08c58d272cccc6f7958ffc42a3cd92073738145c87665b872751d4c04d652a6d","question":"[header] How to shop at holiday craft fairs [title] Seek out local craft fairs. [step] You can find lists of upcoming craft fairs online or in your local newspaper. You may also see advertisements for craft fairs around town.\n\nA. These are likely upcoming gatherings and opportunities to partake in crafts as the holiday approaches. Here are a few craft fairs your neighbors might attend : [substeps] Craft fairs catered to students or non-professionals help plant flowers for toddlers or teenagers [title] Choose a craft fair.\nB. Keep your eyes peeled and try to find a craft fair that meets your needs. [substeps] Oftentimes, organizations plan craft fairs to raise money.\nC. Finding local craft fairs is a chance to meet new people and find fresh experiences without being a tourist. [substeps] Look for fairs that are seasonal and offer a near-annual theme, such as holiday fairs, slumber parties, or seasonal party parties.\nD. Ask if there is a festival coming up. [title] Make an online questionnaire about seasonal crafts.","prompt":"[header] How to shop at holiday craft fairs [title] Seek out local craft fairs. [step] You can find lists of upcoming craft fairs online or in your local newspaper. You may also see advertisements for craft fairs around town.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6370,"error":null,"output":"B"}
{"question_id":"hellaswag:00009702","item_index":203,"attempt":0,"prompt_hash":"a98dc774de3b817ae9458955377f243305f09b5d740a01cebdec24a48d2fecbd","question":"[header] How to tame cowlicks [title] Wet your hair. [step] It is easier to work with wet hair. Once the roots dry, the hair will be set and difficult to style.\n\nA. [substeps] If you plan to work with damp hair, you can water the hair, but use a detangling spray to get your hair as dry as possible. Shampooing your hair while it is wet will make all the tangles out of your hair.\nB. Shampoo your hair and apply a leave-in conditioner. While you can use conditioner every day, it's best to start with wet hair.\nC. Shampooing wet hair takes anywhere from an hour to a very long period. [substeps] Start with your roots and work your way down.\nD. You can style your hair just out of the shower, or wet your hair in the section of the cowlick using a spray bottle. [title] Dry your hair with a hair dryer on medium heat.","prompt":"[header] How to tame cowlicks [title] Wet your hair. [step] It is easier to work with wet hair. Once the roots dry, the hair will be set and difficult to style.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6378,"error":null,"output":"D"}
{"question_id":"hellaswag:00005771","item_index":204,"attempt":0,"prompt_hash":"83744411b708c3e3f6209fcf9db03eeb0e5dd9db383e70478ee32095a8ade187","question":"[header] How to reduce the stigma of ptsd [title] Educate yourself. [step] The first step in reducing social stigma around ptsd is to educate yourself. Once you've learned as much as you can and challenged your own beliefs, you can then begin to educate others and change their attitudes.\n\nA. [substeps] Ask yourself questions like, \" how do i change this stigma about social haptic disorder? \" and \" can i change this stigma about social status? \" [title] Identify your triggers. [step] Once you know more about yourself, work on identifying your triggers.\nB. This will help you better understand and put your answers into concrete terms. [substeps] Get up early every morning and read a book.\nC. [substeps] Ask your peers or college administration how they feel about ptsd. You can then share this information with your friends and family so that your own experiences can be respected.\nD. [substeps] Learn as much as you can about ptsd. Try to understand how this condition and its social stigma affect these individuals, as well as their family and friends.","prompt":"[header] How to reduce the stigma of ptsd [title] Educate yourself. [step] The first step in reducing social stigma around ptsd is to educate yourself. Once you've learned as much as you can and challenged your own beliefs, you can then begin to educate others and change their attitudes.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6372,"error":null,"output":"D"}
{"question_id":"hellaswag:00008092","item_index":205,"attempt":0,"prompt_hash":"fc68cb6d86908d3fc87c257f679a204af75612d80ef7685556041b82d31b9d91","question":"[header] How to get a felony expunged [title] Know what expungement does. [step] Expungement is a process by which you hide an arrest or conviction from your record, which means you will not have to disclose it to potential employers, landlords, or others. Every state manages your criminal record differently, so be sure you understand your state's policy.\n\nA. Make sure you understand your state's guidelines to expunge convicted defendants from your records. [substeps] Some states prohibit expungement for individuals convicted of a felony such as a juvenile offense.\nB. [substeps] In michigan, if you qualify for expungement, your conviction will not be available to the public. In california, an expungement acts as a dismissal.\nC. If you have a felony expunged, you are required to declare your conviction within five years of that conviction. [substeps] Expungement cases generally last between six and ten years.\nD. [substeps] In some states, expungement is permanent without parole. In others, you may be in probation for a period of six years.","prompt":"[header] How to get a felony expunged [title] Know what expungement does. [step] Expungement is a process by which you hide an arrest or conviction from your record, which means you will not have to disclose it to potential employers, landlords, or others. Every state manages your criminal record differently, so be sure you understand your state's policy.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6371,"error":null,"output":"B"}
{"question_id":"hellaswag:00003361","item_index":206,"attempt":0,"prompt_hash":"29d62dd6ce93ce8c80f0f2b18f8db308bb04474534ca2c52cb1e03e2ac44361d","question":"[header] How to make kemplang tunu fish flavored patties (indonesian ) [title] Mix the fish flesh with salt and warm water. [title] Add sago and wheat flour little by little. [step] Mix into the fish flesh to combine.\n\nA. [title] Form the dough into tennis ball sized balls. [title] Press the ball until it is flat, forming patties.\nB. [title] Add the fish to the butter mixture. [step] Crack open a box of bulgur dough and mix it so well that it covers the entire fish.\nC. [title] Dredge the patties in hot water so that excess water dilutes. [title] Peel and chop the cauliflower in small pieces.\nD. It should look like the same consistency when it is almost chunky on the bottom; just read on. [title] Add the sesame oil, saffron and fish sauce to the fish once the oil is fully incorporated and the flesh forms a sort of patty.","prompt":"[header] How to make kemplang tunu fish flavored patties (indonesian ) [title] Mix the fish flesh with salt and warm water. [title] Add sago and wheat flour little by little. [step] Mix into the fish flesh to combine.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6375,"error":null,"output":"A"}
{"question_id":"hellaswag:00005731","item_index":207,"attempt":0,"prompt_hash":"69a6708340433aac5cd6cd98c84b1ae7b694e926e7a3fbb6ea7c56f6223cadfb","question":"[header] How to design a handbag [title] Read the latest magazines. [step] Fashion magazines will obviously have lots of full-page advertisements featuring handbags. But don't forget to look at other kinds of magazines.\n\nA. Lifestyle, celebrity, and even home magazines may show handbags that catch your eye. [substeps] You don't have to limit yourself to the most recent magazines.\nB. Narrow your focus to design cohesive documents and stuff your handbag with them. For example, if you know you plan to go out once a week, expand on how you plan to spend the rest of your week and then specify which ones you will be organizing.\nC. Strong cataloguing means you won't waste time and money just looking at magazines that don't show well. Look for sporting events, which you may wish to attend, and consider organizing the event for them.\nD. Aim for glossy glossy cover ads-because they'll tell you what it does, and how people will use the product! Modern magazines also contain searchable search engines for your favorite designers and the pictures on those pages. [title] Look for scientific papers written by experts.","prompt":"[header] How to design a handbag [title] Read the latest magazines. [step] Fashion magazines will obviously have lots of full-page advertisements featuring handbags. But don't forget to look at other kinds of magazines.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6373,"error":null,"output":"A"}
{"question_id":"hellaswag:00006950","item_index":208,"attempt":0,"prompt_hash":"358d3ee2bb2818657bf7de85048f85774d6478483912ca91d4816dfcb800476a","question":"[header] How to defend a punch [title] Present a profile to your attacker. [step] Turn so one shoulder is pointed at your attacker, instead of facing him straight on. Keep one foot slightly toward him, and the other turned out and slightly behind you.\n\nA. [substeps] You should stand as close as possible with the heel of your knee directly under his body. Start with you dominant foot toward his back, and your non-dominant foot facing forward.\nB. [title] Look them in the eye, but don't deliver the punch. [step] Avoid looking directly at them-this may cause them to think you're running toward them, or they may gauge your stance through body language as well.\nC. Stay light on your feet so you can move quickly in any direction without losing your balance. Your attacker now has a smaller target, with fewer vulnerable areas.\nD. [substeps] Try to be friendly, but not pushy. Say \" hi, i'm claire, what's your name? \" [title] Position the body to make a fist.","prompt":"[header] How to defend a punch [title] Present a profile to your attacker. [step] Turn so one shoulder is pointed at your attacker, instead of facing him straight on. Keep one foot slightly toward him, and the other turned out and slightly behind you.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6375,"error":null,"output":"C"}
{"question_id":"hellaswag:00002356","item_index":209,"attempt":0,"prompt_hash":"5a23cf9b6cccdd02ef979de18ba5c17bc15049ecec62a88e668e67f18f4bb39d","question":"Several people are gathered around a net on the beach. they\n\nA. are playing a game of volleyball.\nB. are engaged in a contest of grabbing and holding a ball.\nC. are engaged in a game of curling.\nD. are holding and playing a game of volleyball.","prompt":"Several people are gathered around a net on the beach. they","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6374,"error":null,"output":"A"}
{"question_id":"hellaswag:00005515","item_index":210,"attempt":0,"prompt_hash":"1ea7139acb7b2a5017eabc1d35a5c8dc4b82458e4a29b8b0952fd269fb0f3c00","question":"[header] How to pick music for a funeral [title] Take the venue into account. [step] If the memorial service is going to take place at a funeral home, you will be able to play almost any music that you like. However, if the service will be held at a religious establishment like a church, then you may be held to the traditions and restrictions of that religion.\n\nA. [substeps] Consult the venue's website for information regarding catholic worship practices and service requests in your area. Check the location of the funeral home beforehand.\nB. Before choosing any music, however, consider the dynamics of the venue before purchasing music. [title] Check if the venue specifies rules and cleanliness.\nC. Be sure to consult with a clergy member to discuss any possible rules in regards to music. [substeps] Even if you can't play particular songs during a service, you may be able to play them while people are entering/exiting.\nD. [title] Take into account the location of the burial location. [step] Bury at an open-casket grave site, or in a bereavement cemetery.","prompt":"[header] How to pick music for a funeral [title] Take the venue into account. [step] If the memorial service is going to take place at a funeral home, you will be able to play almost any music that you like. However, if the service will be held at a religious establishment like a church, then you may be held to the traditions and restrictions of that religion.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6376,"error":null,"output":"C"}
{"question_id":"hellaswag:00005572","item_index":211,"attempt":0,"prompt_hash":"82c3e1c354c19bb5f757637e8f695252847ae9a75268a66a98fa10f83c788a13","question":"[header] How to grieve a parent's unexpected death [title] Take the time you need : [step] At first, you may feel like you are numb, or in shock. Allow yourself the time to just feel sad, or angry, or whatever feelings come. And those feelings are likely to come in waves, slowly.\n\nA. Because grieving is painful, you'll need to allow yourself to process those feelings for a while as well as you allow your body to heal. [substeps] Studies have found that babies who are currently breastfed have an increased risk of developing heart disease, stroke, hypertension, and cancer.\nB. Let yourself feel them as a natural part of the grieving process. [substeps] Though it may not be much of a choice, recognize that you are not alone.\nC. The process of letting go may feel uneven: you'll feel as if you are healing, but then something will trigger more sadness. Give yourself plenty of time for this slow process.\nD. Just give yourself a few days to let them set in. [title] A big part of grieving an unexpected death is being open to other people and new experiences.","prompt":"[header] How to grieve a parent's unexpected death [title] Take the time you need : [step] At first, you may feel like you are numb, or in shock. Allow yourself the time to just feel sad, or angry, or whatever feelings come. And those feelings are likely to come in waves, slowly.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6377,"error":null,"output":"C"}
{"question_id":"hellaswag:00006778","item_index":212,"attempt":0,"prompt_hash":"2e4c9f6d4a011a0a30bba6bae579301fc303c6564e6b9fd1a12a53ef2cd9093f","question":"[header] How to accessorize with a bucket bag [title] Choose a reasonably sized bag. [step] Bucket bags can be a fashionable alternative to a purse, backpack, or briefcase, but you need to make sure your bag can fit everything you need. Take inventory of the largest items in your current bag, and find a bucket bag that is at least half an inch (about 1 cm) larger than your largest items.\n\nA. [substeps] Look at the depth of your current bag, as well. Make sure your bucket bag will be able to expand in all directions to hold what you need.\nB. Think about weight when picking a bucket bag. You want a bag that is firmly packed and compact to accommodate multiple items, and safely holds the things you would need most.\nC. [substeps] Keep all the essentials you need, like keys, basic clothes and accessories, into your bucket bag. You'll need these items when you wear a bucket.\nD. [substeps] There's not much specific size detail about bucket bags. Stick with whatever works for you, while you're at it.","prompt":"[header] How to accessorize with a bucket bag [title] Choose a reasonably sized bag. [step] Bucket bags can be a fashionable alternative to a purse, backpack, or briefcase, but you need to make sure your bag can fit everything you need. Take inventory of the largest items in your current bag, and find a bucket bag that is at least half an inch (about 1 cm) larger than your largest items.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6375,"error":null,"output":"A"}
{"question_id":"hellaswag:00005463","item_index":213,"attempt":0,"prompt_hash":"8ea76c5ee9728637bb30dcd7de79374c8f856b15e7caf666ec86fafcbcc2a1be","question":"[header] How to color hair with food coloring [title] Lighten or bleach your hair, if needed. [step] Food coloring is translucent. This means that it only adds to whatever color is already there.\n\nA. If you want your hair color to be darker, you need to add more food coloring in the same way you would lighten or bleach it. Mix it up with your hair color to achieve a dark, realistic color.\nB. If you have dark hair, the dye may not show up at all. If you have light brown or brown hair, the dye will turn out darker.\nC. For this reason, it's often best to use food coloring instead if you are dyeing your hair light and it has damaged it. [substeps] The beauty of foods lightening (or bleaching) hair is it absorbs the nutrients from the food dye in a more effective manner.\nD. Most people tend to use food coloring once or twice a week, so if you're prone to lightening hair, go ahead. [title] Create a mixture of dyes and artificial hair products.","prompt":"[header] How to color hair with food coloring [title] Lighten or bleach your hair, if needed. [step] Food coloring is translucent. This means that it only adds to whatever color is already there.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6375,"error":null,"output":"B"}
{"question_id":"hellaswag:00008130","item_index":214,"attempt":0,"prompt_hash":"8b44a20f96f6e8c9fa048a19013c4507f594621639ab83f251560221698c85bb","question":"[header] How to teach yourself a language using flashcards [title] Find existing flashcards. [step] Making and customizing your own is the most impactful way to learn, but you can also buy flashcards or find them online. Check retail stores or phone applications.\n\nA. You'll find several excellent kinds of flashcards online that will help you learn a range of languages. [substeps] Ensure your palm, finger, thumb, and index finger are visible before inserting a flashcard.\nB. You can use programs to make your own simple flashcards that you use the same way as you would paper ones. This is at the very least an easy way to start before you tailor your own flashcards to your needs.\nC. They are bound to be able to find flashcards for free or for a significantly lower price. [title] Make flashcards to test out a vocabulary word.\nD. [title] Choose a format and type of flashcard. [step] Either print a one-sentence message or format it by pressing ctrl + f.","prompt":"[header] How to teach yourself a language using flashcards [title] Find existing flashcards. [step] Making and customizing your own is the most impactful way to learn, but you can also buy flashcards or find them online. Check retail stores or phone applications.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6462,"error":null,"output":"B"}
{"question_id":"hellaswag:00001314","item_index":215,"attempt":0,"prompt_hash":"3d00d54a25b3596c6854d84585a14522b6acf20a6e005e3f4a5267d32af4b2d1","question":"A man is standing on a field, playing with a dog and a frisbee. Several people are seated all around, watching. the dog\n\nA. runs across the grass, forcing the man to run after it.\nB. lunges for the frisbee and the man catches it.\nC. finishes the frisbee and the man throws it at the next person walking by.\nD. chases and catches the frisbee numerous times.","prompt":"A man is standing on a field, playing with a dog and a frisbee. Several people are seated all around, watching. the dog","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6463,"error":null,"output":"D"}
{"question_id":"hellaswag:00009523","item_index":216,"attempt":0,"prompt_hash":"68584ab3707fe5debbdaaff1f60ff1a4abce7b1ea596edabe9682f5659724676","question":"[header] How to design an ugly sweater using stickers [title] Set a piece of watercolor paper in either orientation. [step] Decide if you want to show one, two or three sweaters. If you make two, you might include a long-sleeved sweater and a sleeveless vest.\n\nA. [title] Create your design on the underside of the paper, behind the pencil mark, and along the edge. [step] Trace on the paper the design, leaving a border of your design where the pattern will go.\nB. [title] Look at some actual sweaters for ideas. [step] Make the sweater (s) large enough to almost fill the space.\nC. It can be hard to tell the difference between them, so experiment. [title] Trace and cut paper to make the cuff.\nD. If you make three, you might include a short trench coat and a long wool turtleneck. [title] Use a stencil and plenty of pencil to trace your ideas.","prompt":"[header] How to design an ugly sweater using stickers [title] Set a piece of watercolor paper in either orientation. [step] Decide if you want to show one, two or three sweaters. If you make two, you might include a long-sleeved sweater and a sleeveless vest.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6463,"error":null,"output":"B"}
{"question_id":"hellaswag:00008181","item_index":217,"attempt":0,"prompt_hash":"a4e46c94a81ff5de748266ced02be4f757827c2d0cc626e454fa0ff93b06b5a9","question":"[header] How to bleach hair blonde [title] Start with healthy hair. [step] Don't dye or otherwise process your hair in the months leading up to the bleaching. Bleaching your hair will work much better if your hair is relatively strong and unprocessed.\n\nA. Your hair will withstand the rigorous bleaching process better, too. [substeps] Use natural shampoo and conditioner for strong hair.\nB. Hair that is processed the same way too many times and then undermines the color will look mostly the same. [substeps] You can also avoid bleaching by eating a healthy diet and exercising frequently.\nC. Check your hair a week after you've dyed it to see how it reacts before bleaching. Look at your hair every day before you begin.\nD. However, even if you cut your hair short, skip this step. [substeps] If you dyed your hair blonde in the months leading up to the bleaching, skip this step.","prompt":"[header] How to bleach hair blonde [title] Start with healthy hair. [step] Don't dye or otherwise process your hair in the months leading up to the bleaching. Bleaching your hair will work much better if your hair is relatively strong and unprocessed.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6462,"error":null,"output":"D"}
{"question_id":"hellaswag:00001076","item_index":218,"attempt":0,"prompt_hash":"b063b60f1f2211d196bbcd045960b069098aa9667e2aa2e5ed5020588e5c8771","question":"A person is seen standing on a stage playing a violin. the man\n\nA. continues to move his hands all along the instrument and pauses to rest on the notes.\nB. is then seen singing and teaching the class and leads into him playing again.\nC. continues playing the instrument and moving his hands all around the area.\nD. moves his hands up and down on the instrument.","prompt":"A person is seen standing on a stage playing a violin. the man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6462,"error":null,"output":"D"}
{"question_id":"hellaswag:00006942","item_index":219,"attempt":0,"prompt_hash":"154c2893e0a9789853ade4cba4be5f5b87ea0fd03d9c401f75ce8414c3ab3f26","question":"[header] How to front squat [title] Load the bar with weights. [step] Start light with little or no weight if you are a beginner. The proper technique must be learned first, before moving on to lifting the plate-loaded bar.\n\nA. Good weights to start with include dumbbells, plates, and the abba, which is a barbell that sits directly between the nearest plate plate and the side-loaded bar. [substeps] Check your current range of motion for proper degree of lift range.\nB. If you are new to front squatting, you will most likely want to pick a weight that is lighter than you can lift when you are in the air. Without loading the bar, you will do not be able to raise it up and back again.\nC. When you're ready, select plates in the proper weight and slide them onto the bar. Each side of the bar should contain the same amount of weight, which will help to keep it balanced.\nD. [title] Position the bar firmly and with consistent weight. [title] Hold the bar with your dominant hand.","prompt":"[header] How to front squat [title] Load the bar with weights. [step] Start light with little or no weight if you are a beginner. The proper technique must be learned first, before moving on to lifting the plate-loaded bar.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6279,"error":null,"output":"C"}
{"question_id":"hellaswag:00009369","item_index":220,"attempt":0,"prompt_hash":"b20b723372b0d1035cbe794421e207e3832c256a6a4937f065275d260d271349","question":"[header] How to wear culottes [title] Pick culottes that are full but not bulky. [step] The unusual shape of culottes can quickly go from trendy to frumpy if the fit is bulky or too structured. Choose a lighter fabric that has a little swing to it, like jersey, chiffon, silk, or satin.\n\nA. The legs should be loose-fitting and drape naturally. [substeps] Avoid stiffer heavy materials like tweed, wool, and heavy linen.\nB. Wear culottes with short tops and narrow bottoms. A top that is too loose gives the impression of size.\nC. [substeps] If you look fashionable and casual, a culottes with a looser fit/fold will do to make you look slimmer. [title] Don't wear caps or wide-brimmed hats.\nD. You will also want a cut that is flatter and tighter. [substeps] Let the culottes hang loosely on your body.","prompt":"[header] How to wear culottes [title] Pick culottes that are full but not bulky. [step] The unusual shape of culottes can quickly go from trendy to frumpy if the fit is bulky or too structured. Choose a lighter fabric that has a little swing to it, like jersey, chiffon, silk, or satin.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6280,"error":null,"output":"A"}
{"question_id":"hellaswag:00004082","item_index":221,"attempt":0,"prompt_hash":"3037e2b667f3021c66a4cf3e6465261d8ff65eec9cd2be10183bacece20fa706","question":"[header] How to waist train without a corset [title] Practice forearm plank. [step] The transverse abdominus muscles act like an inner-girdle, wrapping around the sides of your waist. A great exercise for working the transverse abdominus is a basic forearm plank.\n\nA. [substeps] Stand with your feet shoulder-width apart and your hands facing away from you. Press your palms together and pull with enough force to lock these muscles into a tight rope shape.\nB. [substeps] Lay on your back on the floor, with your arms extended in front of you, palms flat on the floor. Your arms should be by your sides, with the palms facing each other.\nC. Begin with your forearms and knees on the ground, with your arms shoulder-width apart. Press up into a forearm plank, keeping your body in one straight line.\nD. Lay on your back on the floor with your arm resting over your head. Turn your head in a clockwise direction with your torso extended in front of you.","prompt":"[header] How to waist train without a corset [title] Practice forearm plank. [step] The transverse abdominus muscles act like an inner-girdle, wrapping around the sides of your waist. A great exercise for working the transverse abdominus is a basic forearm plank.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6280,"error":null,"output":"C"}
{"question_id":"hellaswag:00007744","item_index":222,"attempt":0,"prompt_hash":"d9554961909778233faedb6ec2f39bbb55f085ecfac8c32525fcb7d273582253","question":"[header] How to buy children pyjamas as a gift [title] Get the right size. [step] When choosing the size, you might consider that measurements are average. And if the child is smaller or bigger than his coevals, you might want to adjust the size.\n\nA. [title] To select the proper design the best option would be to ask the parents of the child what he likes. [step] Depending on the age, child's interests might be in toys, cartoons, films, books, sports, video games, etc.\nB. As a general rule, take measurements from the smallest to the largest. This will help calculate the portion of the party that is needed, and will make it easier to feed and store the rest of the party.\nC. [title] Avoid pre-washed pyjamas with labels, details and chemicals. [step] Consider purchasing pre-washed pyjamas because they are more gentle, make it easier to wash.\nD. [title] Buy eco-friendly pyjamas. [step] As a gift owner and a parent of a child whose daughter slept on the sofa where you've put a pillow and blanket, think carefully.","prompt":"[header] How to buy children pyjamas as a gift [title] Get the right size. [step] When choosing the size, you might consider that measurements are average. And if the child is smaller or bigger than his coevals, you might want to adjust the size.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6279,"error":null,"output":"A"}
{"question_id":"hellaswag:00004490","item_index":223,"attempt":0,"prompt_hash":"99ad5d89c7b24ac638ad69a52a2a1cc87a219bb23493017f36de8b6e8c2654d4","question":"[header] How to play volleyball [title] Choose your players. [step] Ideally, you want at least 6 players for a competitive game. However, if you're playing with a group of friends, keep the same number of players on each side.\n\nA. It is also good to have 3 or more people to make game decisions. [substeps] The point of the game of volleyball is to quickly and smoothly guide the players around the court.\nB. This way, you'll actually be able to toss through hoops together. [title] Determine who's responsible for shooting the ball.\nC. This can allow for more control, motion, and more dribbling opportunities. [substeps] When choosing players, it may help to consider who will play a particular area at different times of the game, though-don't split up your group of players.\nD. [substeps] You should have 2 rows of players, with the first row closest to the net and the back row closest to the rear boundary of the court. The player in the back right corner will serve the ball.","prompt":"[header] How to play volleyball [title] Choose your players. [step] Ideally, you want at least 6 players for a competitive game. However, if you're playing with a group of friends, keep the same number of players on each side.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6279,"error":null,"output":"D"}
{"question_id":"hellaswag:00005080","item_index":224,"attempt":0,"prompt_hash":"a2e2fbef1fe8046fe93fea99a1e7bedbab6ce9f0801e2bc16e1b2f43e99ff8c6","question":"[header] How to trade gold coins for cash [title] Understand the difference between bullion and numismatic values of gold. [step] Any scratches, nicks, stains, and fingerprints can significantly reduce their value, so take every precaution to protect your gold coins. Avoid cleaning your gold because cleaning decreases its value.\n\nA. [substeps] If you're simply purchasing gold, you shouldn't keep cleaning it. Your gold coins can wear down very quickly and could potentially become out of commission.\nB. [title] Avoid being tempted to trade your gold coins. [step] Private collectors frequently ask for gold coins and can sometimes feel jealous that the coin is being traded in exchange for cash.\nC. [substeps] Use water and a soft cloth to clean dust and debris from surface silver objects. Washed items must be rinsed with soap and water, in order to avoid the possibility of your item getting torn by dirt.\nD. [substeps] As with any valuable assets, keep them locked up and secured. Consider using plastic coin protectors to prevent damage from handling.","prompt":"[header] How to trade gold coins for cash [title] Understand the difference between bullion and numismatic values of gold. [step] Any scratches, nicks, stains, and fingerprints can significantly reduce their value, so take every precaution to protect your gold coins. Avoid cleaning your gold because cleaning decreases its value.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6723,"error":null,"output":"D"}
{"question_id":"hellaswag:00006564","item_index":225,"attempt":0,"prompt_hash":"3ae06185d5c26522e9ff67ad99903f9f8b6aea7c1f49b57ba07e2a396b9c15ea","question":"[header] How to dress in simple alternative fashion [title] Decide on a style. [step] You don't need to decide on a label; go with what catches your eye. However, just make sure you have a pretty good idea of what you want.\n\nA. [title] Consider what \" texas style \" means to you. [step] Texas style is a standard style, but you should go with something more...straight-forward, elegant, fun, and versatile.\nB. As always, try looking back in the mirror and removing any items that clash with your utilitarian look. You can refine your choices by researching online and by word-of-mouth.\nC. It might help to take some extra time to think, and you might benefit from doing a shoe-browsing tutorial too. If your budget is high, a friend might be able to design.\nD. Print or cut our pictures from the internet and magazines of styles you like to keep you inspired. Paste them to your wall, or create a little fashion notebook and put them in there, if you think it would look awesome with your décor.","prompt":"[header] How to dress in simple alternative fashion [title] Decide on a style. [step] You don't need to decide on a label; go with what catches your eye. However, just make sure you have a pretty good idea of what you want.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6722,"error":null,"output":"D"}
{"question_id":"hellaswag:00001458","item_index":226,"attempt":0,"prompt_hash":"d7ea33ee53abd41d29199b698c7918bd422d909508fbc8483e00f162e2ca2111","question":"A person wearing gloves is seen spreading food onto a plate followed by a large pastry and spreading butter on it. the person\n\nA. bakes a piece of cake followed by multiple also together.\nB. continues making layers and then undoes the cake and shows off the solid cake.\nC. is then seen eating a bowl of food and helping others.\nD. continues to spread food all around while the camera pans around chasing the food around with a spoon.","prompt":"A person wearing gloves is seen spreading food onto a plate followed by a large pastry and spreading butter on it. the person","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6728,"error":null,"output":"D"}
{"question_id":"hellaswag:00004744","item_index":227,"attempt":0,"prompt_hash":"664200012137d05850817e254e5593cb92fe2a73add2467b397fd661e93d48aa","question":"[header] How to remove an item from a list in foursquare [title] Open up foursquare. [title] Search for the item you'd like to remove from the list. [step] You can tap the \" lists \" tab at the bottom of the screen, tap the list, then tap the item from the list.\n\nA. [title] Tap the info tab under the business' name and category/city. [title] Scroll up until you see the buttons for \" add to list \" and \" check in \", \" rate \" and \" leave a tip \" just above the \" similar to (x and y business) \" links line.\nB. [title] Swipe up and down on the item. [step] The item will slide right, making it much easier to remove.\nC. [title] Tap \" shop \" next to the item you'd like to remove. [title] Find and tap \" get rid of \" next to its name.\nD. [substeps] The item is an item with an icon of a winking face (or cartoon character) on it. If any item's value isn't represented, tap the item to remove it from the list.","prompt":"[header] How to remove an item from a list in foursquare [title] Open up foursquare. [title] Search for the item you'd like to remove from the list. [step] You can tap the \" lists \" tab at the bottom of the screen, tap the list, then tap the item from the list.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6722,"error":null,"output":"B"}
{"question_id":"hellaswag:00008209","item_index":228,"attempt":0,"prompt_hash":"0faa32ba5ff114ff349c5d6183d4dab048d6a733406428d0771bbf877ed9ac8a","question":"[header] How to dunk an oreo cookie [title] Obtain a box of oreo cookies from the local grocery store or gas station. [step] Make sure they are the original cookies and not the new vanilla oreos. While at the store, pick up a gallon or half gallon of milk.\n\nA. Use a pair of shallow feet to lift up the box to remove the cracked cookies. [substeps] If you can't get any oreo cookies, you can also buy regular oreo cookies instead.\nB. ( 2 l-4 l) if you think you are going to spill much of the milk, go with the gallon (4 l), but if you are a neat and tidy eater, go with the half gallon (2 l). If you already have milk, make sure it's fresh.\nC. Do not remove the cookies when the milk is poured in, though; you'll be holding them for a long time and using up the milk in the process. [substeps] Follow the directions on the box of cookies for the correct amount of time.\nD. Place the oreo cookie in the milk container and then dunk your cookie, immersing it in the milk. Allow your cookie to soak in the milk overnight to allow the oreo to soften (waiting too long for it to absorb).","prompt":"[header] How to dunk an oreo cookie [title] Obtain a box of oreo cookies from the local grocery store or gas station. [step] Make sure they are the original cookies and not the new vanilla oreos. While at the store, pick up a gallon or half gallon of milk.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6722,"error":null,"output":"B"}
{"question_id":"hellaswag:00008165","item_index":229,"attempt":0,"prompt_hash":"b27313fde73c61aef32dffbb09ac6e242dfcd72ef18c3524bc2648d856e9d9d6","question":"[header] How to decorate your porch for halloween [title] Create a jack-o'-lantern. [step] Purchase a pumpkin and carve it into an iconic halloween jack-o'-lantern. If you're really looking to turn heads, you can buy multiple pumpkins and carve more than one.\n\nA. Purchasing stencils can help you carve an intricate design even if you aren't normally artistically inclined. [substeps] To learn how to carve your jack-o'-lantern read how to carve a pumpkin.\nB. Jack-o-lantern is available in various sizes and shapes. [substeps] For more details, see: how to decorate a jack-o-lantern for halloween.\nC. [substeps] In one example, you could have several people make the frogs, animals, or stars (someone who knows all the steps). The idea is to have four people making a jack-o-lantern about the size of an avocado.\nD. [substeps] While you should only use a pumpkin as a christmas sign, adding a jack-o-lantern could give your porch a fancy, festive feel. Be sure to take pictures on a mount near your house or over on the mantle or railing.","prompt":"[header] How to decorate your porch for halloween [title] Create a jack-o'-lantern. [step] Purchase a pumpkin and carve it into an iconic halloween jack-o'-lantern. If you're really looking to turn heads, you can buy multiple pumpkins and carve more than one.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6721,"error":null,"output":"A"}
{"question_id":"hellaswag:00004987","item_index":230,"attempt":0,"prompt_hash":"28fe6f1b01af0f8cf4c3637d42fe0a1f65d47915d99001f81e7fc9823e69796a","question":"[header] How to create a wedding memory book [title] Decide the scope of your wedding memory book. [step] Do you want to just document the wedding-related events (the proposal, dress selection, bridal shower, wedding day, etc. ) or would you like to include memorable events throughout your relationship and engagement? This determines the length and photo selection of your book.\n\nA. This map will help you choose a good book. Always make sure that an event you want to have relatable usually falls within the scope of your wedding memory book.\nB. [title] Look through the proofs of your wedding photos and mark the ones that stand out to you. [step] Go with your gut feeling and err on the side of inclusion--you can sort out the bad ones later.\nC. [substeps] For example, be sure to include pictures of your wedding events to assign to your book. [title] Write the hopes and dreams of each person in your book.\nD. [substeps] It may help for you to consider what works best with the image of the bride and groom. The book may reflect certain moods, seasons, and places or experiences which essential elements become important.","prompt":"[header] How to create a wedding memory book [title] Decide the scope of your wedding memory book. [step] Do you want to just document the wedding-related events (the proposal, dress selection, bridal shower, wedding day, etc. ) or would you like to include memorable events throughout your relationship and engagement? This determines the length and photo selection of your book.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6722,"error":null,"output":"B"}
{"question_id":"hellaswag:00007577","item_index":231,"attempt":0,"prompt_hash":"aed304b76d87098401d56a5d2891372dda22a3fdc5679af1377cc0a1deadf616","question":"[header] How to deep clean a house [title] Clean your dishwasher. [step] Dishwashers can usually be cleaned by running a cleaning cycle. Before doing so, however, use a paper towel to remove any loose bits of food from the bottom of the dishwasher.\n\nA. [title] Dump a good amount of water into the dishwasher. [step] Add either a drop of dish soap or a non-soapy dish detergent.\nB. [substeps] Usually, to run a cleaning cycle, you place a cleaning solution in the dishwasher and then let it run. Refer to your dishwasher's specific instruction manual for the precise directions for your washer.\nC. [substeps] If your dishwasher has a spray bottle, remove the top and spray the entire bottom with hot water. Once the water has completely cooled down, wipe any food residue from the top of the dishwasher with a soft cloth.\nD. These can then dry in the same manner as paper towels. [substeps] The only difference this process makes is that without any food dripping from the bottom of the dishwasher, the dishes are likely to get dirty.","prompt":"[header] How to deep clean a house [title] Clean your dishwasher. [step] Dishwashers can usually be cleaned by running a cleaning cycle. Before doing so, however, use a paper towel to remove any loose bits of food from the bottom of the dishwasher.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6721,"error":null,"output":"B"}
{"question_id":"hellaswag:00002583","item_index":232,"attempt":0,"prompt_hash":"70b72aa6bef5f6cf8b90d01ea7819d470c8bb2780263fd08d6f7210fc1deddbb","question":"A man wearing a hat in standing in a workshop with a harmonica in hand. he\n\nA. puts the harmonica down after adjusting the settings of the neck.\nB. begins to play the harmonica and break dance as he describes the high octave octave of the song.\nC. talks and he shows how to form your mouth so you can play the harmonica.\nD. is dressed in a red kid's outfit with a bandanna over it.","prompt":"A man wearing a hat in standing in a workshop with a harmonica in hand. he","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6723,"error":null,"output":"C"}
{"question_id":"hellaswag:00001876","item_index":233,"attempt":0,"prompt_hash":"064cd6bc60b1150ff47553b09385562ffa1b3e8c19f3f011d61552793a5b040a","question":"A lady gives a man a piece of paper. The man fills a white bowl with food. the man\n\nA. slides the bowl towards the lady.\nB. blows on the food.\nC. takes a piece of bread and fries it.\nD. washes dishes using soap, water, and a peeler while talking.","prompt":"A lady gives a man a piece of paper. The man fills a white bowl with food. the man","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6722,"error":null,"output":"A"}
{"question_id":"hellaswag:00002013","item_index":234,"attempt":0,"prompt_hash":"16051cd39008aabf36da48539b0196a61bdd34fc7d8898d741cb5b9fbc899b13","question":"A man is shown flipping a violin in his hands, playing a popular song on his violin. The camera shows the violinist from various angles playing the popular song and not breaking concentration. the camera\n\nA. shows a different ending to the song.\nB. continues sliding down to show the violin and he is cutting up the skin further.\nC. pans all around the player and shows him playing the complete song.\nD. zooms in to show the finished peelings and the ending title screen.","prompt":"A man is shown flipping a violin in his hands, playing a popular song on his violin. The camera shows the violinist from various angles playing the popular song and not breaking concentration. the camera","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6722,"error":null,"output":"C"}
{"question_id":"hellaswag:00003094","item_index":235,"attempt":0,"prompt_hash":"ba2633a887ea7a62972b2378f898941cfb9224e851f14787bff90800ff490ded","question":"A man shoots a basket from the corner of the free throw line. The man walks over to retrieve the ball and shoots a basket from outside the key. the man\n\nA. rolls into the basket then walks to the high railing to collect another basket.\nB. shoots the ball around a corner only leaving the basket behind.\nC. shoots the ball and throws the ball into the net, it lands on the grass.\nD. shoots a basket from the top of the key.","prompt":"A man shoots a basket from the corner of the free throw line. The man walks over to retrieve the ball and shoots a basket from outside the key. the man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6721,"error":null,"output":"D"}
{"question_id":"hellaswag:00001583","item_index":236,"attempt":0,"prompt_hash":"47476389ca6ab12f50b13646ce0a92f3f1d659619563601233c2cb2aa88cd855","question":"After that, they take off in motion in the water connected by a cord to a speed boat. the video\n\nA. is then shown again in detail as the other boy checks the pedals on the speed boat.\nB. ends when they both fall int the water.\nC. shows surfer follows by one walking along a sea and jumping in the water.\nD. ends with a photograph of naked women in bed and white text scrolling down the middle.","prompt":"After that, they take off in motion in the water connected by a cord to a speed boat. the video","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6721,"error":null,"output":"B"}
{"question_id":"hellaswag:00004391","item_index":237,"attempt":0,"prompt_hash":"4e30cb829ad42619e82b9cae641bdcb0d052d1fde5976702bda06fa5bcdfb1e3","question":"[header] How to deal with thick leg hair [title] Allow the hair to grow. [step] If you want to deal with hair growth on your legs, allowing your hair to grow actually makes surface hair removal treatments more effective. Because the hair strands will be longer, there will be more for a shaver or razor to latch onto.\n\nA. [substeps] It's a good idea to shave at night rather than during the day. Shaving at night will leave your legs feeling smoother.\nB. [substeps] As a general rule, the longer you let your hair grow, the thicker it will become. Trying to move the hair or playing with it too much will result in harsh stubble in your layered cut and eventual hair loss.\nC. [title] Wrap your hair in an elastic band and feed the ends of the elastic to the hair growth band. [step] The elastic band will hold the hair tight and allow you to avoid cutting the ends.\nD. Some experts suggest that this is a good time to only allow hair to grow if you want to take a time-out or to work on different things (optional). [substeps] This is a good time to work on old towel towels or paper towels.","prompt":"[header] How to deal with thick leg hair [title] Allow the hair to grow. [step] If you want to deal with hair growth on your legs, allowing your hair to grow actually makes surface hair removal treatments more effective. Because the hair strands will be longer, there will be more for a shaver or razor to latch onto.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6719,"error":null,"output":"A"}
{"question_id":"hellaswag:00007092","item_index":238,"attempt":0,"prompt_hash":"a170c73bfe155603b6f856d7d12321734304981164713463b8e78efb79e17d47","question":"[header] How to fiberglass speaker boxes [title] Clean your work area thoroughly. [step] It is highly undesirable to allow dust to collect on the fiberglass resin. If you're working in the trunk or any part of the car that has carpeting, it advisable to remove it.\n\nA. [title] Lay plastic covering around the area where you want the speaker enclosure. [step] Cover any surface where you do not want resin dust.\nB. [substeps] Remove scuff marks, dents, or other damages. Place pieces of fiberglass resin on boards, clamp, and other hard surfaces.\nC. Clean it with a sponge or cloth first so all traces of dirt is removed. [substeps] When cleaning your speakers, follow these recommendations: do not touch your speakers, as this can cause serious damage.\nD. Remove all dust from floor boards, carpet, chairs, and areas where the panel is installed. [substeps] If there are any air vents or ventilation holes on the inside of the speaker box, replace those or open them so a draft can circulate.","prompt":"[header] How to fiberglass speaker boxes [title] Clean your work area thoroughly. [step] It is highly undesirable to allow dust to collect on the fiberglass resin. If you're working in the trunk or any part of the car that has carpeting, it advisable to remove it.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6719,"error":null,"output":"A"}
{"question_id":"hellaswag:00002818","item_index":239,"attempt":0,"prompt_hash":"fc7a59e30bd3db869059da56f2aacc07e05ab40890357dfa10f1c03eb667bccc","question":"A person is running on an excerise machine. the camera\n\nA. shows the final result.\nB. zooms in to the feet.\nC. cuts to a club scene.\nD. pans onto trees and a car driving past.","prompt":"A person is running on an excerise machine. the camera","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6720,"error":null,"output":"B"}
{"question_id":"hellaswag:00004580","item_index":240,"attempt":0,"prompt_hash":"26b7160932103619c2c137c63532a9c55ae373e918028c8a66c0b1575752a252","question":"[header] How to diagnose and treat gingivitis in cats [title] Look at their gums. [step] To check your cat's gums, sit down with the cat. Tip their head back, and part their lips with a finger so that you can see their gums.\n\nA. [substeps] You can also gently slide your finger in between their upper teeth. Your finger should be just under the gum line, and you should not see any visible blood vessels.\nB. Your cat's gums are upper and lower than those of a normal cat. [substeps] If your cat's gums are whiter than normal, it likely has gingivitis.\nC. Then, look for white gums, a pink gum line, and stripes or markings. These are all signs of gingivitis.\nD. Typically, gingivitis will cause redness and swelling around the gum line. You may also notice some plaque build-up on their teeth.","prompt":"[header] How to diagnose and treat gingivitis in cats [title] Look at their gums. [step] To check your cat's gums, sit down with the cat. Tip their head back, and part their lips with a finger so that you can see their gums.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6719,"error":null,"output":"D"}
{"question_id":"hellaswag:00006423","item_index":241,"attempt":0,"prompt_hash":"09f5ad46e689bdeb8de4aec039ac73e3c0901484f5a2ab81ac961d6821c37155","question":"[header] How to curve a soccer ball [title] Line up your kick. [step] Approach the ball at an angle. You will plant your non kicking foot about 12-16 inches away from the ball.\n\nA. [substeps] Set up slightly off center from where you would if you wanted to kick the ball straight. A right footed kicker should approach the ball from left to right and a left footed kicker from right to left.\nB. Then, hold your right (kicking) foot out in front of you, with your shin slightly bent and your right knee tilted to align your upper leg. Remember that the shins should be right under your front knee and your upper thigh should be held out behind your upper leg.\nC. [substeps] Also, position your dominant foot slightly forward and close. This will make it much easier to determine the right angle for the throwing motion.\nD. This is the point where you're going to make contact. Make sure that your toe is pointing straight back, and that you've placed your weight correctly.","prompt":"[header] How to curve a soccer ball [title] Line up your kick. [step] Approach the ball at an angle. You will plant your non kicking foot about 12-16 inches away from the ball.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6718,"error":null,"output":"A"}
{"question_id":"hellaswag:00003872","item_index":242,"attempt":0,"prompt_hash":"50b4d326d639138646de3d5c03aefc74dea8d52d0470a4bf200da7a650518d06","question":"[header] How to change a mortise lock cylinder [title] Open door partially to reveal the mortise lock cover plate at the edge of the door. [step] Remove the two retaining screws and set aside. [title] Note the position of the keyway as it relates to the cylinder.\n\nA. [step] If, for instance, a half turn tells you the full profile of the old lock, you must approximate the main combination and the starting and ending positions, regardless of what's on your paint job. [title] Look at the half turn shape used by all locks.\nB. [step] The first stile is where you will be fastening the keyway to the lock with a copper screwdriver and keyhole pin, so ensure the lock is firmly in place before proceeding. [title] Make sure the correct keyway channel is located between the floor and door jamb.\nC. [step] Most likely it is at the bottom (six o'clock). [title] Remove the cover plate from the edge of the door and back out the set screw that is preventing the cylinder from turning.\nD. [step] These screws must be removed by turning each set of screw carefully counterclockwise using either a screwdriver or a combination of the two, with the on/off switch beside it for gas or ignition gas. [substeps] The wires must be flush with the surface of the dial.","prompt":"[header] How to change a mortise lock cylinder [title] Open door partially to reveal the mortise lock cover plate at the edge of the door. [step] Remove the two retaining screws and set aside. [title] Note the position of the keyway as it relates to the cylinder.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6719,"error":null,"output":"C"}
{"question_id":"hellaswag:00004139","item_index":243,"attempt":0,"prompt_hash":"0b1dae1c7bfc6c8e60e7852a0e92b1163f97e3597aeedeb4bedb6b9eb9435e5a","question":"[header] How to settle an argument with your wife [title] Stay in the present. [step] Arguments are often the result of underlying tensions in a relationship. If you want to settle an argument effectively you must be willing to communicate with your wife.\n\nA. This means staying in the present moment and avoiding discussing past conflicts. [substeps] In an argument, you might be tempted to bring something up from the past.\nB. When your wife is upset, she often grabs her husband by the collar, trying to get his attention by hitting and spanking him until she calms down. This sort of behavior leads to hurt feelings, stress and anger and even physical arguments.\nC. You should focus on the present moment and trust her to tell you exactly what you need to do to resolve the issues you are having. [substeps] Try keeping a journal or diary about what is going on in your marriage.\nD. When a positive resolution comes along, speak with your wife face-to-face rather than monopolizing the conversation or answering your texts. You want her attention at all times, but don't disappear and leave her to fend for herself.","prompt":"[header] How to settle an argument with your wife [title] Stay in the present. [step] Arguments are often the result of underlying tensions in a relationship. If you want to settle an argument effectively you must be willing to communicate with your wife.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6719,"error":null,"output":"A"}
{"question_id":"hellaswag:00005910","item_index":244,"attempt":0,"prompt_hash":"d4dd9b29d7c0ce369c10f573365b1f442b7f296acef439a24ad879fef6ef0130","question":"[header] How to start writing [title] Target the cause. [step] Almost anything could be discouraging you from writing whether it is mental, emotional or physical. When you begin to write, jot down the thoughts that come to your mind even if they aren't related to what you need to write about.\n\nA. Try to pinpoint where your thoughts came from as well. Also reflect on what else you're already prone to.\nB. Sometimes it helps to have a companion source to speak to. This can be a friends or family member or yourself.\nC. [substeps] Try to think of something you are doing that most potential readers would find interesting and appealing. Think of a reason why you want to write from that angle.\nD. This exercise is known as free or random association will give you a clue towards what the problem is. Here are some common reasons to why people have trouble writing and some suggestions on how to overcome the hurdle.","prompt":"[header] How to start writing [title] Target the cause. [step] Almost anything could be discouraging you from writing whether it is mental, emotional or physical. When you begin to write, jot down the thoughts that come to your mind even if they aren't related to what you need to write about.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6720,"error":null,"output":"D"}
{"question_id":"hellaswag:00007045","item_index":245,"attempt":0,"prompt_hash":"e7c53443fcc3da27182fe6b669e0d5de30d26c166bcca6fdeb9b68b5949a2828","question":"[header] How to drive a stick shift with a broken left foot [title] Get in the vehicle. [substeps] Lean your crutch up against the side of the car, right behind the driver's side door. Unlock the door while maintaining your balance, and carefully slide into the driver's seat.\n\nA. Start wrapping your left foot around the front wheel to bring the crutch up and in. This will give you the best possible grip.\nB. Your feet should be in contact with your hazard board, and your knees should be bent at 90 degrees as you lean into the car. Using your left leg, lean and center your weight onto your right leg.\nC. To avoid further injury, it is advised to slide in with your backside first. If you are using a second crutch, place it on the passenger seat or behind the driver's seat.\nD. [title] Mount your snare and turn it on. [substeps] Grip both of the rear tires while rotating the snare counterclockwise.","prompt":"[header] How to drive a stick shift with a broken left foot [title] Get in the vehicle. [substeps] Lean your crutch up against the side of the car, right behind the driver's side door. Unlock the door while maintaining your balance, and carefully slide into the driver's seat.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6721,"error":null,"output":"C"}
{"question_id":"hellaswag:00006500","item_index":246,"attempt":0,"prompt_hash":"ca032e1ba9503d41c3ac57b062517546a5d75c417cbaf24bb02c7dc21538328c","question":"[header] How to make money as an elementary school student [title] Ask for an allowance. [step] Many children earn a weekly allowance. This allowance is to pay for everyday things like buying food or going out with friends.\n\nA. It might be nice to extend it on a regular basis. The amount you earn per week doesn't necessarily have to be that huge depending on the age of the child and its needs.\nB. Ask the teacher if they have a reasonable allowance you can take advantage of. If you don't receive one, ask if you can limit your allowance to how much you can earn.\nC. Having an allowance can help you manage your money well-even if you don't have an allowance. [title] Teach your children about the value of money and the benefits of work.\nD. If you don't already earn an allowance, ask you parents if they would be willing to start giving you an allowance. [title] Ask for a raise in your allowance.","prompt":"[header] How to make money as an elementary school student [title] Ask for an allowance. [step] Many children earn a weekly allowance. This allowance is to pay for everyday things like buying food or going out with friends.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6635,"error":null,"output":"D"}
{"question_id":"hellaswag:00000525","item_index":247,"attempt":0,"prompt_hash":"690bd32f4ee7715d06d281212dc068247573f94ea151251e805f4492b83d4f52","question":"When the cake is all done the camera pans the whole cake so the details can be seen and once again we see the smaller scale cake and the toy. a white screen\n\nA. appears with black letters below and we see a white screen with white words on it at the bottom of it and it says how to get it from the register with an online purchase screen.\nB. appears with his logo on the left, and all of his detailed information for his shop and contact information on the right.\nC. appears and words appear on the screen that say \" jackpot eels \".\nD. appears and then starts moving, the drawings again.","prompt":"When the cake is all done the camera pans the whole cake so the details can be seen and once again we see the smaller scale cake and the toy. a white screen","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6636,"error":null,"output":"B"}
{"question_id":"hellaswag:00004152","item_index":248,"attempt":0,"prompt_hash":"ef950f4b4a5da614f88ee1da6e07878ece0aa9c37f3bf7e94de6e370b4a1e6b6","question":"[header] How to comfort someone who lost a loved one [title] Choose the appropriate time to talk. [step] In all of your interactions, make sure that bereaved person is in the proper mindset to have a discussion. They may be particularly upset or dealing with another critical issue.\n\nA. Ask if it is a good time to talk before going on. It also usually a good idea to talk with them one-on-one, when possible.\nB. [substeps] Always begin a conversation with someone else, only people you trust. It's important that bereaved persons find the right chance to talk to you, especially people who have the same loved one as them, so wait until the person is sleeping or is feeling better before speaking.\nC. [substeps] Avoid scheduling a time where the bereaved person is busy or stressed. Try to prepare yourself for this discussion by taking the risk that a time that usually seems like the minimum will be enough.\nD. Avoid being late because you let the person know, or like to interrupt the conversation. Make time to talk when you have both time to think about the issues surrounding the loss of a loved one and time to get some fresh perspective.","prompt":"[header] How to comfort someone who lost a loved one [title] Choose the appropriate time to talk. [step] In all of your interactions, make sure that bereaved person is in the proper mindset to have a discussion. They may be particularly upset or dealing with another critical issue.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6635,"error":null,"output":"A"}
{"question_id":"hellaswag:00006010","item_index":249,"attempt":0,"prompt_hash":"08e5eab63c4eebed679f9f7cbbfb816303ba3216ab2f8d9ead73975d510c6764","question":"[header] How to enjoy a caribbean cruise with children [title] Look for a cruise line that offers family-friendly accommodations and activities geared towards children. [substeps] Research cruise lines such as royal caribbean, norwegian and carnival. These lines have ships that include waterslides, rock climbing walls, miniature golf and arcades.\n\nA. Compare the prices, family size, age and living arrangements of each ship. If your kids don't already have friends familiar with the ship or to whom they are glad to have the cruise, consider buying one from them.\nB. Reserve a cabin that is large enough to accommodate your family. Cabins are notoriously small on most cruise ships, so you may need to ask for an upgrade.\nC. [title] Make sure you're not the only passenger on board. [substeps] Many ship owners and cruise companies will connect to friends or family on board.\nD. Different cruise lines have options for children to board as well as sunbathing. Looking for a cruise line that offers attractions like concerts, lounges, museums, and flea markets are also options.","prompt":"[header] How to enjoy a caribbean cruise with children [title] Look for a cruise line that offers family-friendly accommodations and activities geared towards children. [substeps] Research cruise lines such as royal caribbean, norwegian and carnival. These lines have ships that include waterslides, rock climbing walls, miniature golf and arcades.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6635,"error":null,"output":"B"}
{"question_id":"hellaswag:00005636","item_index":250,"attempt":0,"prompt_hash":"29ec49108b8b6df774c126d729a8744fb86beab5754a869505e4a9d0c5edc19a","question":"[header] How to ask your girlfriend to stop talking to a guy [title] Explain your feelings with \" i \"-statements. [step] As this is a delicate situation, phrasing things using \" i \"-statements can help reduce tension as you place the focus on your feelings. Begin with \" i feel...\" and state your feeling.\n\nA. Speak in a firm voice that focuses the conversation on your feelings, not the actual how you feel. [substeps] This is a good way to keep distance between yourselves without allowing her to get defensive.\nB. [substeps] For example, \" i am a fragile person. \" this statement will help take the focus off of the conversation so you don't sound accusatory.\nC. Essentially, this phrase can help your girlfriend know exactly what you don't like about her. [substeps] For example, state: \" i feel jealous when you talk behind your girlfriend's back.\nD. Then, explain the behavior that led to that feeling and why you feel the way you do. [substeps] Don't come into the situation and say something like, \" it's hurtful that you're so flirtatious with your ex and it makes me insecure.","prompt":"[header] How to ask your girlfriend to stop talking to a guy [title] Explain your feelings with \" i \"-statements. [step] As this is a delicate situation, phrasing things using \" i \"-statements can help reduce tension as you place the focus on your feelings. Begin with \" i feel...\" and state your feeling.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6635,"error":null,"output":"D"}
{"question_id":"hellaswag:00009932","item_index":251,"attempt":0,"prompt_hash":"2f25c0b053f2853d84cd1230b6a233492086eb21d6498f104116b4913dd54eec","question":"[header] How to choose the right partner in life [title] Identify your deal-breakers. [step] Ask yourself which personality traits, habits, or circumstances you can't accept in a partner. Be prepared to stop seeing someone if any of your deal-breakers are present.\n\nA. [substeps] Determine if you find any unappealing traits, like a bad-faith attitude, that you want out of a partner. [title] Consider whether you're dating if you're single.\nB. [substeps] Separate your true deal-breakers from traits that might be annoying, but aren't the end of the world. For instance, poor hygiene might be a true deal-breaker for you.\nC. If you're having some difficult or complicated relationships, cut off ties and keep your sex life to yourself, and only interact with that person. [substeps] For example, you might say, \" we really aren't as emotionally compatible as we used to be, and we're not looking for more in-depth relationships.\nD. Try to avoid people who are overly competitive, are always rude, or don't respect your boundaries. Also, think about whether or not you stand out to other people and people and reevaluate your goals.","prompt":"[header] How to choose the right partner in life [title] Identify your deal-breakers. [step] Ask yourself which personality traits, habits, or circumstances you can't accept in a partner. Be prepared to stop seeing someone if any of your deal-breakers are present.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6899,"error":null,"output":"B"}
{"question_id":"hellaswag:00005549","item_index":252,"attempt":0,"prompt_hash":"22697e9cb5b310ee73d6b88543421f17b7849ee2c043564025519bad4bd0e35e","question":"[header] How to introduce your baby to peanuts [title] Talk to your baby's doctor. [step] Before you introduce peanuts to your baby, talk to your pediatrician. Your doctor can confirm that it is safe for your baby to try peanuts, especially if they have other food allergies or eczema (also known as atopic dermatitis).\n\nA. [substeps] Answer any questions your pediatrician may have about family history with peanuts. If you or the baby's other parent has a peanut allergy, the baby may also have one.\nB. [substeps] Ask your pediatrician for information on medications your baby may take and treatment options for their condition. You may be responsible for paying an additional expense if you get peanuts in your baby's food.\nC. Your pediatrician will also be able to diagnose any allergies or allergy disorders your baby may have and may recommend appropriate care. [title] Have a variety of foods that the baby eats throughout the day.\nD. [substeps] The doctor can also give you an overview of the peanuts package to help you further discuss what you can expose your baby to during allergy season and how the process works properly. [title] Keep your baby's mouth clean.","prompt":"[header] How to introduce your baby to peanuts [title] Talk to your baby's doctor. [step] Before you introduce peanuts to your baby, talk to your pediatrician. Your doctor can confirm that it is safe for your baby to try peanuts, especially if they have other food allergies or eczema (also known as atopic dermatitis).","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6899,"error":null,"output":"A"}
{"question_id":"hellaswag:00003534","item_index":253,"attempt":0,"prompt_hash":"ce1db5751662b74f4bbbbf8f50cde19dbdbf49d2a27bf3b17574b7a2c4ef0ada","question":"[header] How to create custom lists in excel [title] Open microsoft excel on your computer. [step] Excel is the spreadsheet tool of the microsoft office productivity suite. It is available on both windows and mac operating systems.\n\nA. [substeps] Excel uses data from the cloud that is needed to generate spreadsheets in excel. You can see the contents of excel on a separate context panel in the upper right section of excel.\nB. [substeps] If you do not have excel installed on your computer, you can connect it to a windows pc as long as the computer still has blank columns in the file. Windows xp excel focuses on creating a list by stored spreadsheets.\nC. [substeps] If you do not already have excel installed, run a background check to make sure it is running as usual. [title] Go to the filing tab on the left.\nD. [title] Launch excel's custom lists editor. [step] How you open the custom lists editor will vary depending on your operating system and software version of excel.","prompt":"[header] How to create custom lists in excel [title] Open microsoft excel on your computer. [step] Excel is the spreadsheet tool of the microsoft office productivity suite. It is available on both windows and mac operating systems.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6899,"error":null,"output":"D"}
{"question_id":"hellaswag:00006794","item_index":254,"attempt":0,"prompt_hash":"f962d3a730b49352095e8fce19e0acabcd8aaee4d0e26140bd9bf9a8fc4b6642","question":"[header] How to conk hair [title] Use a lye-relaxer if you have coiled or curly hair. [step] Traditionally, conking hair was done by making a homemade relaxer with lye. Lye-relaxers are strong enough to straighten hair that is curly or coiled.\n\nA. To make one, use the tip of a spoon. Place the bristle head of the spoon into the palm of your hand.\nB. They may also help straighten out the split ends. [substeps] Be aware that the extra thickness help hair relax and become smooth.\nC. But hair preserved in the lye bath can be treated with at least a couple of days' worth of hair spray or leave-in shampoo. Lye's high oxalate content is important for hair's longevity.\nD. [substeps] Lye is a strong chemical that can burn your skin and hair if too much is applied or it sits on your hair too long. Always use lye-relaxer with caution, preferably with help from a friend.","prompt":"[header] How to conk hair [title] Use a lye-relaxer if you have coiled or curly hair. [step] Traditionally, conking hair was done by making a homemade relaxer with lye. Lye-relaxers are strong enough to straighten hair that is curly or coiled.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6900,"error":null,"output":"D"}
{"question_id":"hellaswag:00001333","item_index":255,"attempt":0,"prompt_hash":"c7d9a16a8f19acb8e8859cf29a91232f55e22a66bb622ad6fac64aad32708bd8","question":"A woman is riding a horses through a wooded area. Other people watch as she talks to them. he\n\nA. horse nods and neighs as the woman smiles.\nB. ends the video with him tightening his rope.\nC. discusses her technique with her, then she is up on a horse.\nD. saddles up an attack horse for a ride.","prompt":"A woman is riding a horses through a wooded area. Other people watch as she talks to them. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6899,"error":null,"output":"A"}
{"question_id":"hellaswag:00002048","item_index":256,"attempt":0,"prompt_hash":"4e61638c782501545876cc23178428e2f14f71358bd1d34cb92ba844dde89e14","question":"We see the mower from the point of view of the mower. We see the boy start the lawn mower again. The boy mows the lawn. we\n\nA. then see the boy walking with the mower.\nB. see the note on the screen.\nC. see the house with a van outside the screen.\nD. see a new mower in the yard.","prompt":"We see the mower from the point of view of the mower. We see the boy start the lawn mower again. The boy mows the lawn. we","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6511,"error":null,"output":"A"}
{"question_id":"hellaswag:00000553","item_index":257,"attempt":0,"prompt_hash":"bf1181479849076af7a6e4ef89b98e9fc2306d296daadc9211ff126bd77272f0","question":"The two girls hug each other. The two girls sit in front of the oven licking their mixing spoons. the two girls\n\nA. tear up and speak to each other as they continue to eat their lunches.\nB. begin to scrape the top off the potatoes.\nC. remove the baking sheet from the oven.\nD. take a glass o milk then chug it.","prompt":"The two girls hug each other. The two girls sit in front of the oven licking their mixing spoons. the two girls","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6510,"error":null,"output":"C"}
{"question_id":"hellaswag:00002488","item_index":258,"attempt":0,"prompt_hash":"b77ba3b35eb903d1f003362a23d97f7bfada01a012e77ae7f7fe9cce9bed56a0","question":"A few people are seen standing around on a basketball court. the group\n\nA. then begins playing pulled up by a rope with an instructor.\nB. of girls apply logos and words to the hoop.\nC. then begins shooting baskets and passing to one another.\nD. spreads out into teams, holding sticks to run up and throws.","prompt":"A few people are seen standing around on a basketball court. the group","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6510,"error":null,"output":"C"}
{"question_id":"hellaswag:00004371","item_index":259,"attempt":0,"prompt_hash":"7a727d1ff355cdac9ea229a8fa6b45677708114cc8a1d10d87a55fad845f3ebe","question":"[header] How to get your children up in the morning [title] Open the curtains to let the sunshine in. [step] Light, particularly natural light, stimulates our brains to release serotonin, which helps us to wake up and feel alert. It is a crucial step in the process of waking up, and it will help your child to overcome the early-morning drowsiness that they are feeling.\n\nA. [substeps] Never go to sleep late at night unless you are absolutely sure that the window is open enough. If you let in natural light, it can disrupt the natural circadian rhythm of your sleep cycle.\nB. Open the shades to expose more light to let the light come in. [substeps] You can also turn on natural bright lamps to do this.\nC. [substeps] Open the blinds or curtains in your child's room about 15 minutes before you want them to get out of bed. The sunshine will help them to wake up naturally.\nD. [substeps] As you're opening the curtains to let light in, you should pause halfway across your bedroom and focus on your window. This gives your child an opportunity to engage their senses in the room.","prompt":"[header] How to get your children up in the morning [title] Open the curtains to let the sunshine in. [step] Light, particularly natural light, stimulates our brains to release serotonin, which helps us to wake up and feel alert. It is a crucial step in the process of waking up, and it will help your child to overcome the early-morning drowsiness that they are feeling.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6504,"error":null,"output":"C"}
{"question_id":"hellaswag:00004655","item_index":260,"attempt":0,"prompt_hash":"39aac9ffb1f627e5c55e0a7001594bf7b4bf1665efb0204f07bd18063d107b70","question":"[header] How to cook mongolian beef [title] Cut the mongolian beef across the grain into bite-size or smaller) pieces. [title] Combine the marinade ingredients in the medium mixing bowl. [step] Make sure to add ingredients in the order they are listed.\n\nA. [title] Cover the pan and refrigerate for 4 hours. [title] Heat oil in a place non flammable and turn one medium-sized beef piece in a pan about 2 inches (5 cm) away from any heat source for about a minute.\nB. [title] Add the meat pieces and coat well with the marinade. [step] Then, let the meat marinate, undisturbed for thirty or forty minutes.\nC. Stir the beef in the mixing bowl and let it sit for approximately a minute to allow the marinade to develop and enter the beef properly. [title] Pour or spray the marinade over the beef as indicated in the recipe.\nD. [title] Season the beef, if it is in season. [step] Add seasonings as they are listed.","prompt":"[header] How to cook mongolian beef [title] Cut the mongolian beef across the grain into bite-size or smaller) pieces. [title] Combine the marinade ingredients in the medium mixing bowl. [step] Make sure to add ingredients in the order they are listed.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6509,"error":null,"output":"B"}
{"question_id":"hellaswag:00003456","item_index":261,"attempt":0,"prompt_hash":"fac07b022d1a1c6b80069ac0fa834600a762ee48904156293e4283539c6991ec","question":"[header] How to whiten skin with fruits [title] Drink citrus juice. [step] Drinking a citrus juice like orange juice may help to inhibit uv induced pigmentation. [title] Use citrus to make a mask.\n\nA. [step] There are dozens of different ways to create a mask. Many citrus fruits have blended chemicals that degrade and preserve a face.\nB. [step] You can make your own mask by mixing 3 cups (710 ml) of oranges , 2 cups (480 ml) of water, and 2 tablespoons (30 ml) of raw coconut oil together in a bucket or jar. [substeps] Most coconut oil may cause irritation and irritation at first, but over time, this inflammation will go away.\nC. [step] Citrus fruits contain vitamin c, which promotes collagen production (necessary for the creation of new skin cells), is a powerful antioxidant and has been shown to inhibit skin darkening due to uv exposure. [substeps] Lemons tend to work best, but any citrus fruit will work, including oranges, limes, and grapefruit.\nD. [step] You can make a mask by using dry skin. Soak a toothbrush, a small container, or a sponge in liquid citrus juice.","prompt":"[header] How to whiten skin with fruits [title] Drink citrus juice. [step] Drinking a citrus juice like orange juice may help to inhibit uv induced pigmentation. [title] Use citrus to make a mask.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6508,"error":null,"output":"C"}
{"question_id":"hellaswag:00003816","item_index":262,"attempt":0,"prompt_hash":"5e5c84ae5cf09557a471f2817fb91226153a8c75ee9c3595df1625364f3c7919","question":"[header] How to smoke turkey legs [title] Smoke turkey legs using a smoker. [substeps] Preheat the smoker to 200 to 220 degrees fahrenheit (93 to 104 degrees celsius). Rinse the turkey legs under cool water and pat them dry with paper towels.\n\nA. Once they cool, coat both sides with a layer of oil for a smoother and quicker smoking option. Place turkey legs in a microwaveable container.\nB. Coat the turkey legs, as well as under the skin, with your favorite seasonings. Place the turkey legs in the smoker and cook for about 4 hours.\nC. Insert the turkey legs into the middle of the smoker and insert the kindling plug, lighter or smoking stick, cord, or 5-gallon fuel bottle into the smoker. [substeps] You'll need to insert the wood plugs into the bottom of the smoker just before smoking starts.\nD. Put the turkey legs in the smoker. [title] Cure the turkey legs inside the refrigerator.","prompt":"[header] How to smoke turkey legs [title] Smoke turkey legs using a smoker. [substeps] Preheat the smoker to 200 to 220 degrees fahrenheit (93 to 104 degrees celsius). Rinse the turkey legs under cool water and pat them dry with paper towels.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6503,"error":null,"output":"B"}
{"question_id":"hellaswag:00007263","item_index":263,"attempt":0,"prompt_hash":"6e891c0601be18a358da5c77be601a7141adcd589ba1fb1b74411b8e9df3181f","question":"[header] How to handle an employee with substance abuse issues [title] Watch out for unusual behavior. [step] There are an array of symptoms that might point to a person being on drugs, because it will depend upon the drug and its properties. For some, drugs will make them drowsy, relaxed, and happy.\n\nA. Others might worsen their symptoms with recovery. [substeps] If you see a friend, family member, colleague, or coworker going to a drug store, do not assume the individual is on drugs by their appearance or demeanor, or by their behavior.\nB. Other drug-related responses include insomnia and headache, which can be indicative of substance abuse. If you notice any of these symptoms, that includes : [substeps] Trouble remembering details regarding the drug or its effects on you mood, reasoning, or decision.\nC. Other people who abuse substances tend to be very energetic and erratic in their behavior. You may notice that the person is always on the move but never seems to actually achieve anything.\nD. For others, eye injuries, convulsions, or other issues can also indicate drug addiction. [substeps] It is important to know that these things are merely behavior, not things for you to take measures to prevent.","prompt":"[header] How to handle an employee with substance abuse issues [title] Watch out for unusual behavior. [step] There are an array of symptoms that might point to a person being on drugs, because it will depend upon the drug and its properties. For some, drugs will make them drowsy, relaxed, and happy.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6502,"error":null,"output":"C"}
{"question_id":"hellaswag:00009245","item_index":264,"attempt":0,"prompt_hash":"cb8b427d96800448b2b9e795c598652cc5ffed7aebd213f4f3e8b60a4b2db0fc","question":"[header] How to read wind barbs [title] Read the wind direction. [step] The \" flagpole \" or directional vector of a wind barb shows you which way the wind is blowing. Make sure that north is oriented up.\n\nA. Facing east for a counterclockwise wind. Going east or west for a counterclockwise wind from east to west.\nB. North indicates east, west for south, and vice versa. [substeps] If the wind rises to 44 °, north actually indicates north, but it does not show up at \" east.\nC. [substeps] North might indicate a north wind to an east, if your wind is ranging south, south, northwest, or west, then north probably indicates a west wind to your north-west. South might indicate a west wind to east, if your wind is going south, northwest, or south, and west indicates a west wind to your west.\nD. Just like a map or a compass rose, a wind barb plot is typically drawn with north at 0 ° and south at 180 °. You can learn the direction of the wind by reading the angle of the barb vector in degrees.","prompt":"[header] How to read wind barbs [title] Read the wind direction. [step] The \" flagpole \" or directional vector of a wind barb shows you which way the wind is blowing. Make sure that north is oriented up.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6502,"error":null,"output":"D"}
{"question_id":"hellaswag:00003956","item_index":265,"attempt":0,"prompt_hash":"87d500159885020dfc73431b6b52503ac62b4e7f47467464f40b47bf7bed3ecd","question":"[header] How to cut baby hair [title] Make sure that your baby is in a good mood. [step] Cranky, fussy babies will move around a lot, which can make cutting difficult. A happy baby may still squirm, but not to the extent of flailing arms and legs.\n\nA. [substeps] If your baby moves too much, they may struggle to stay upright. Feeding them, spinning, getting on all fours, and even holding them too close can make your baby move in a different direction and upset someone or lose themselves.\nB. [title] Cut away the excess hair on the side of the head. [step] Your newborn baby's head should look round and raised when it is in its crib or sling.\nC. [title] Begin the cutting procedure fairly early on. [step] It may seem like it might be easier as you're getting your baby out of the bath, but make sure that you make it to the allotted time and then prepare for next time.\nD. More importantly, you want your baby to associate haircuts with good memories-not memories of being hungry, tired, or just cranky. [substeps] Make sure that your baby is fed and rested.","prompt":"[header] How to cut baby hair [title] Make sure that your baby is in a good mood. [step] Cranky, fussy babies will move around a lot, which can make cutting difficult. A happy baby may still squirm, but not to the extent of flailing arms and legs.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6502,"error":null,"output":"D"}
{"question_id":"hellaswag:00003907","item_index":266,"attempt":0,"prompt_hash":"76ebb7f3ff54372b76d9ce79920b538cd07863dea6fff46c59a0c284e1191cbd","question":"[header] How to dress sexy but casual [title] Combine your casual clothing with sexier pieces. [step] Creating a balance between sexy and casual is all about mixing up lengths, styles, and fits. You can wear things that drape for a casual look combined with something tight to add a bit of sex appeal, or you can show off your legs with a sexy skirt while wearing a more modest top.\n\nA. [substeps] Combine a short bottom, such as a mini shirt, with a turtleneck or long sleeved top. Pair a low cut or open back shirt with a fashionable pair of wide-leg pants.\nB. To create a balance, buy sexier, and more revealing tops. [substeps] When wearing a modest top, wear it under your clothes or just on top.\nC. [substeps] Some lingerie to select are form-fitting trousers, loose and sexy dresses, and culottes. Look for sweatpants and flats to add a little sexiness to your look.\nD. [substeps] Choose with the hotel or destination that you plan to go to. For example, a business suit may be more modest than a beach top paired with sneakers for a classic, casual look.","prompt":"[header] How to dress sexy but casual [title] Combine your casual clothing with sexier pieces. [step] Creating a balance between sexy and casual is all about mixing up lengths, styles, and fits. You can wear things that drape for a casual look combined with something tight to add a bit of sex appeal, or you can show off your legs with a sexy skirt while wearing a more modest top.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6503,"error":null,"output":"A"}
{"question_id":"hellaswag:00003729","item_index":267,"attempt":0,"prompt_hash":"82e02ff77c5ec8845fa5cad043e12058f6ed1df2402411134d29daee58f6f406","question":"[header] How to bend electrical pvc pipe on the fly [title] This is a favorite impromptu installation utilizing the gas barbecue. [step] Most homes have one and they get nice and hot with indirect flames. [title] Start by firing it up and letting it get warm.\n\nA. [title] Put it in the oven for 5-10 minutes. [step] Attach the pvc fuse or bond and e-(or connect it with 2-3 phillips screws) and let it set.\nB. [title] Before you have any drama occurring begin performing work into the approved position in the ground, with your feet on the ground at all times. [title] Minimize the electrical shock the falling pipe.\nC. [step] Place one or two pvc pieces on fire. The pieces of a common tin can can can withstand a lot of direct heat.\nD. [title] Some models come with a slot on either side that happens to allow most small pipe (1/2 \" to 3/4 \") to slide right in. [title] Slip in the pvc pipe and slowly rotate it with the lid closed.","prompt":"[header] How to bend electrical pvc pipe on the fly [title] This is a favorite impromptu installation utilizing the gas barbecue. [step] Most homes have one and they get nice and hot with indirect flames. [title] Start by firing it up and letting it get warm.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6503,"error":null,"output":"D"}
{"question_id":"hellaswag:00008745","item_index":268,"attempt":0,"prompt_hash":"87bab6a4d913f53b4d11684f9a0cf27b69ad52f55183686c9f3714099996fce8","question":"[header] How to pintuck jeans [title] Cuff the bottom of each leg to your ideal length. [step] Roll up the cuff of each pant leg to adjust the cuff length. This will also give you an idea of what the cuff will look like when the rolled-up part is removed.\n\nA. [substeps] Cuff the bottom of each pant leg at 45-degrees. This will be the spot where you will cuff your jeans, according to the length guidelines for your version of skinny jeans.\nB. [substeps] You can cuff all the legs of your jeans without unraveling the cuff inch or two. Rolling down your pant leg means you'll have to work on reducing the length of the cuff [title] Fold the pant leg in half at the hip.\nC. Go slowly-there's no need to rush the process. Most shops will give you a rule of thumb when it comes to fitted jeans.\nD. Use a piece of white chalk to mark the bottom of the trench formed by rolled fabric. [substeps] You may want to put these pants on and check out the length in a mirror.","prompt":"[header] How to pintuck jeans [title] Cuff the bottom of each leg to your ideal length. [step] Roll up the cuff of each pant leg to adjust the cuff length. This will also give you an idea of what the cuff will look like when the rolled-up part is removed.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6503,"error":null,"output":"A"}
{"question_id":"hellaswag:00005958","item_index":269,"attempt":0,"prompt_hash":"0de3579b16775b79562460210c3f3c9b2c2b262333d6bc60040bba37f2a72784","question":"[header] How to become a \" horse whisperer \" [title] Respect that horses are powerful and complex animals. [step] If you do not take the time to properly respect horses, you should not attempt to become a horse whisperer. This means putting the needs of the horse before your own desires.\n\nA. Keep in mind that horses won't be able to control your wishes. They will show off and press you and right away you are going to have a rocky relationship.\nB. [title] Do not expect the horse's reactions to be the same as if you were simply being a horse whisperer. [step] Horses are somewhat complex animals in nature.\nC. Horses are great at learning and communicating; they love being around other people, their best friends, or their parents. A famous horse whisperer was a highlighters man.\nD. You have to be able to respect the emotions and desires of the horse. This is an attitude and it inflects each interaction you have with the horse.","prompt":"[header] How to become a \" horse whisperer \" [title] Respect that horses are powerful and complex animals. [step] If you do not take the time to properly respect horses, you should not attempt to become a horse whisperer. This means putting the needs of the horse before your own desires.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6506,"error":null,"output":"B"}
{"question_id":"hellaswag:00002835","item_index":270,"attempt":0,"prompt_hash":"aaa9b9644fa94d6eebd5baa91b4cd8432df44439ef3e9925c05debb312c833eb","question":"As they continue to play, the referee begins to skate back and forth on the side in circles. after several starts and finishes\n\nA. , a man in a red shirt skates through.\nB. , the games keeps going and the goalie is successful in blocking the puck each and every time.\nC. , the the team gives all their credits for the video.\nD. , the clock strikes and the players are congratulated for their hard work, along with many others.","prompt":"As they continue to play, the referee begins to skate back and forth on the side in circles. after several starts and finishes","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6505,"error":null,"output":"B"}
{"question_id":"hellaswag:00002630","item_index":271,"attempt":0,"prompt_hash":"39f5da64936fcc401174cc47c069d338c879ed1a2ad1c478240330163b023689","question":"Once the gates are down they are on the move riding through the course. Group after group starts racing and moving along really fast. the peddle\n\nA. as fast as they can trying to stay ahead of one another.\nB. the momentum getting in the start.\nC. is part of the wrong side of the barn.\nD. elegantly through the course.","prompt":"Once the gates are down they are on the move riding through the course. Group after group starts racing and moving along really fast. the peddle","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6505,"error":null,"output":"A"}
{"question_id":"hellaswag:00003282","item_index":272,"attempt":0,"prompt_hash":"5b49306f7797e33b4daf1458bc71f9ec335353bd935c7d27e7ecf0af0039d73b","question":"[header] How to adopt a natural beauty routine for sensitive skin [title] Determine your skin type. [step] Not all skin types are sensitive. Many are oily, others are dry, and still others are considered \" normal.\n\nA. \" knowing your skin type will help you determine the best options for your lifestyle for the right amount of time and space. [substeps] Oily skin needs to be considered an environmental condition, not reflect it on other skin types.\nB. \" consult your skin type in some detail and learn the best way to moisturize your skin. There are a number of some makeup companies, including benzophenone and magicmerchants, that offer a significant range of makeup options from light shades, lowlights, and deep-colors to more subtle shades.\nC. \" you can determine your level of skin sensitivity by looking for specific symptoms: itching, tightness after using soap, burning, or stinging. [substeps] Darker-colored skin is more prone to sensitivity.\nD. \" you need a certain sort of skin to be able to produce natural beauty. There are differences in your skin type, but think about what is fine and what is not.","prompt":"[header] How to adopt a natural beauty routine for sensitive skin [title] Determine your skin type. [step] Not all skin types are sensitive. Many are oily, others are dry, and still others are considered \" normal.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6504,"error":null,"output":"C"}
{"question_id":"hellaswag:00006621","item_index":273,"attempt":0,"prompt_hash":"e237c0f4dd3d6c446c2beddfc27bea85bd2f03b2f08f2f52a40587521e0f0c1c","question":"[header] How to use an infant nebulizer [title] Put the nebulizer on the baby. [step] An infant will inhale the medication in a mist provided by the mask. Read and follow the manufacturer's instructions and any additional advice provided by your child's doctor.\n\nA. [title] Hold the device on your baby, then follow these instructions : [step] Place a clean dish towel over the device. Insert a small amount of tsp into the nebulizer and over your baby's head.\nB. [substeps] Some babies need only 1 dose every 3-4 hours. [title] Watch for trouble breathing and a ringing sound in their ears.\nC. [substeps] The child should breathe normally while the mask is on. Only four hours have passed since the child first began using the nebulizer, so if someone else has opened it, then your child will usually breathe through the mask.\nD. Study pictures or watch a training video to learn how best to use the nebulizer. Most nebulizers are easy to set up.","prompt":"[header] How to use an infant nebulizer [title] Put the nebulizer on the baby. [step] An infant will inhale the medication in a mist provided by the mask. Read and follow the manufacturer's instructions and any additional advice provided by your child's doctor.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6504,"error":null,"output":"D"}
{"question_id":"hellaswag:00006187","item_index":274,"attempt":0,"prompt_hash":"9f81830b6c9ad3fcd0f51160e07c18eb27a26c252ae9caef28d94bcb1abf4b31","question":"[header] How to make makeup from eyeshadow [title] Find an old eyeshadow (or buy a cheap one). [step] It is fine either in the form of loose or compacted powder, but don't use any in gel form. [title] Scoop the eyeshadow into a bowl.\n\nA. [step] Fill it up almost to the rim of the bowl, but don't overfill. You may have to press on the clumps a little to help them settle.\nB. [step] You'll also want to add two drops of food coloring or rubbing alcohol. [title] Dip a cotton ball into the eyeshadow, for a very small amount.\nC. [step] Fill the bowl 3/4 of the way full of liquid. [title] Begin to roll the eyeshadow along the crease.\nD. [step] Use the back of a spoon to mash it into a fine, lump-free powder. [substeps] To give the lipstick some shimmer, add a little shimmering eyeshadow to the main color you have chosen.","prompt":"[header] How to make makeup from eyeshadow [title] Find an old eyeshadow (or buy a cheap one). [step] It is fine either in the form of loose or compacted powder, but don't use any in gel form. [title] Scoop the eyeshadow into a bowl.","gold":"D","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6504,"error":null,"output":"B"}
{"question_id":"hellaswag:00006811","item_index":275,"attempt":0,"prompt_hash":"27864511fc9dbc22700a2a7b42284ca701e916894ea6c2ecec3bcfa2847f5929","question":"[header] How to forget a network on android [title] Open your android's settings app. [step] The settings icon looks like a gray gear or wrench icon on most android devices. [title] Find the wireless & networks heading.\n\nA. [step] This button is located in the upper-left corner of your screen. Tap it to open the cellular settings dialog box.\nB. [step] The wi-fi button is located on the far left corner of the app window, and it will display a list of all the wi-fi networks nearby. Tap on the wi-fi heading to view the wi-fi networks in your region.\nC. [step] On the main menu of your android device, tap the network gear icon to bring up the wireless network settings menu. [substeps] If the network heading doesn't appear in the settings menu, you can find it by swiping down on the radio button at the top of the screen, or by looking at \" settings \" at the bottom of the screen.\nD. [step] This section will be at the top of your settings menu. [substeps] Depending on your device model and current software, you might have to tap a connections tab at the top of your screen.","prompt":"[header] How to forget a network on android [title] Open your android's settings app. [step] The settings icon looks like a gray gear or wrench icon on most android devices. [title] Find the wireless & networks heading.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6503,"error":null,"output":"D"}
{"question_id":"hellaswag:00005201","item_index":276,"attempt":0,"prompt_hash":"55db733965d5ae36eafb114b3389fc2c312e99d830af12fbd1f24e3a11efdb2e","question":"[header] How to make a burning laser [title] Select a diode strength and color. [step] The strength of a diode will be listed in microwatts (mw). The color of the diode will be determined by its wavelength (measured in nanometers, nm).\n\nA. You can search online for diode strength and color charts. [substeps] Most lasers will be of 14 point, but you can also find lasers that are below 8 point.\nB. [substeps] The direction of the lasers will show. The colors will indicate your lasers' heat element composition and the opacity of your lasers.\nC. You can find diode strength readings on the internet. Read the instructions on the dash website to calculate the strength of a diode.\nD. A wavelength of 650 nm corresponds to a red laser, a wavelength of 405 nm corresponds to a blue laser, and a green laser has a wavelength around 520 nm. Green lasers are the most expensive, followed by blue.","prompt":"[header] How to make a burning laser [title] Select a diode strength and color. [step] The strength of a diode will be listed in microwatts (mw). The color of the diode will be determined by its wavelength (measured in nanometers, nm).","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6503,"error":null,"output":"D"}
{"question_id":"hellaswag:00005285","item_index":277,"attempt":0,"prompt_hash":"a7cdff9d211c1ffe963c6b39496e435e252e7feecfa23ad877c994de84c2ae21","question":"[header] How to make aftershave [title] Choose your main astringent. [step] An astringent is a substance that shrinks body tissues-in this case, your pores. When they shave, most people use warm water, which opens up the pores.\n\nA. Take your astringent and ask an attendant at a health store where you can pick it up in liquid form. [substeps] Astringents in-store are slightly more expensive than anything you can buy online.\nB. You might consider : [substeps] Rubbing some witch hazel or aloe vera oil onto affected areas. Massaging some astringents on blocked pores.\nC. Using an astringent in your aftershave is important because it reverses that process. Astringent also sterilizes any cuts or nicks you accidentally give yourself while shaving.\nD. You can use either honey or, in some cases, a reverse astringent, such as citrine. [substeps] Honey tends to help rid your skin and hair of oil.","prompt":"[header] How to make aftershave [title] Choose your main astringent. [step] An astringent is a substance that shrinks body tissues-in this case, your pores. When they shave, most people use warm water, which opens up the pores.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6503,"error":null,"output":"C"}
{"question_id":"hellaswag:00008042","item_index":278,"attempt":0,"prompt_hash":"a55adaaf8f03ef3522d9e0f5fc7bc74fd109dd0979b693952c2acfe9778d2550","question":"[header] How to make apple jam [title] Peel and chop your apples. [step] Wash your apples and peel them. Core the apples to remove their stems.\n\nA. Trim the apples into smaller pieces to use as jam. Use 3 to 4 large apple chunks in the jam.\nB. Some may have chunks of skin if you cut them off. You'll need about two peeled apples for the total jam.\nC. Carefully dice the apples finely so they're no larger than 1/4 \" cubes. Place all of the apples in a large 4-cup measuring cup.\nD. From the core, you should cut off all the small, stringy flesh. You should also grate them on a separate cutting board.","prompt":"[header] How to make apple jam [title] Peel and chop your apples. [step] Wash your apples and peel them. Core the apples to remove their stems.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6539,"error":null,"output":"C"}
{"question_id":"hellaswag:00006390","item_index":279,"attempt":0,"prompt_hash":"93a7820c2372c631219ed37c1468685bc449014b908ddf32bb22e156dad0a8da","question":"[header] How to overcome trypophobia [title] Understand trypophobia. [step] People who have trypophobia suffer from an irrational fear of clusters of holes. Some examples of triggers include bubbles, lotus flowers, and aerated chocolate.\n\nA. Each individual may feel more anxious on the first day of middle school if they know that people in their classes will think about them all day long. Some people may panic or end up having a panic attack at the first signs of anxiety.\nB. Trypophobes report nausea, trembling, and severe anxiety when confronted with their triggers. Unlike some phobias that may dominate people's thoughts, trypophobia seems only to affect sufferers when they see holes.\nC. Common triggers include : [substeps] Birds overhead branches telephone poles workplace work unexpectedly [title] Recognize symptoms of hypophobia. [step] Anxiety triggers are persons experiencing negative thoughts and feelings during certain activities.\nD. This fear is usually about exponential equations and can influence your choices. If you're trying to overcome a phobia of clumps, you must build up an courage-based phobia.","prompt":"[header] How to overcome trypophobia [title] Understand trypophobia. [step] People who have trypophobia suffer from an irrational fear of clusters of holes. Some examples of triggers include bubbles, lotus flowers, and aerated chocolate.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6537,"error":null,"output":"B"}
{"question_id":"hellaswag:00005456","item_index":280,"attempt":0,"prompt_hash":"de3c33d996e563f1ad495e69baf8f1333e7171f31e915ebfc26c2dac2c1c252e","question":"[header] How to wash your face with rice water [title] Choose your rice. [step] You can prepare rice water with any type of rice, though white, brown, and jasmine rice are common options. If you already own rice, whatever kind of rice you have on hand will work.\n\nA. [title] Put 1/2 cup (92.5 g) of rice into a bowl. [step] If you want to make large quantities of rice water, you can increase the amount of rice that you use, as long as you remember to also increase the water.\nB. [substeps] Soy rice works well, as will turmeric, cloves, and jalapeño. [title] Pour rice water into a mug or cup.\nC. [substeps] New clients and those who have just started out might choose white rice, but it's less likely to taste good due to its other flavors. [title] Add a few drops of garlic and red pepper to your rice water.\nD. If you don't already own rice water, you can buy it at your local grocery store or online. [substeps] The rice doesn't need to be whole grains, which means it just needs to be dried.","prompt":"[header] How to wash your face with rice water [title] Choose your rice. [step] You can prepare rice water with any type of rice, though white, brown, and jasmine rice are common options. If you already own rice, whatever kind of rice you have on hand will work.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6537,"error":null,"output":"A"}
{"question_id":"hellaswag:00000042","item_index":281,"attempt":0,"prompt_hash":"de50f484ed881807844218d7ba3389bbb10f3b5db90d193603efe0ec7255d162","question":"The boys talk, laugh and tease each other before beginning a game of beer pong across a long table using a triangle shaped array of clear cups. one of the ping pongs\n\nA. is knocked upside down, which causes the other teens to give each other high fives.\nB. tries to rise but dikes instead.\nC. rush walks up in front of the boys, removes a small piece of paper from the bottom of the table, puts a hand on the machine and wrestles with buttons, turns the machine off and pushes a button inside to turn the game back on.\nD. lands in the cup and one of the boys begins to drink the beer.","prompt":"The boys talk, laugh and tease each other before beginning a game of beer pong across a long table using a triangle shaped array of clear cups. one of the ping pongs","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6538,"error":null,"output":"D"}
{"question_id":"hellaswag:00000742","item_index":282,"attempt":0,"prompt_hash":"ecd0d8f052a242f9fee30752058179d60f914a81ad0484e21b93b269c47f33c1","question":"A boy stands holding a stick. A man throws pieces of wood into a pile. Chairs at the camp site are shown. a man\n\nA. hangs dead leaves on a tree.\nB. sits in front of a wall calendar.\nC. reels in a jack o lantern.\nD. walks towards the camera talking.","prompt":"A boy stands holding a stick. A man throws pieces of wood into a pile. Chairs at the camp site are shown. a man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6537,"error":null,"output":"D"}
{"question_id":"hellaswag:00007952","item_index":283,"attempt":0,"prompt_hash":"8bdd6cc9fecf99e43de429210b2fd91c5cec1e8435dc4861a719a6f8dc4b93c6","question":"[header] How to help your chronically ill toddler exercise [title] Make an appointment with your child's pediatrician. [step] Your toddler's doctor can help you understand your child's needs, strengths, and limitations. They may also be able to recommend basic exercises that are safe and appropriate for your child's age, physical condition, and overall health.\n\nA. [substeps] Avoid asking your doctor any leading questions during the appointment. Your child is supposed to know their' doctor' better.\nB. Tell the doctor about your toddler's health history, any current symptoms, and any medications they are taking. [substeps] Depending on your child's condition, your pediatrician may recommend medications to help make physical activity easier and safer.\nC. [substeps] If you child's doctor does not have a thorough list of approved exercises for their recommendation, you should ask if a physical therapy specialist is in your area. [title] Suggest encourage that your toddler exercise outdoors.\nD. [substeps] If your toddler's medical needs do not involve exercise or your toddler's health, your doctor may be able to suggest alternative exercises that could ease your toddler's symptoms. [title] Ensure the cramps are minimized.","prompt":"[header] How to help your chronically ill toddler exercise [title] Make an appointment with your child's pediatrician. [step] Your toddler's doctor can help you understand your child's needs, strengths, and limitations. They may also be able to recommend basic exercises that are safe and appropriate for your child's age, physical condition, and overall health.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6583,"error":null,"output":"B"}
{"question_id":"hellaswag:00005423","item_index":284,"attempt":0,"prompt_hash":"c08c37f0302d7471129ebb73762e0a265f16c9daa5d0a83bdae3a38494c5267b","question":"[header] How to get beautiful, glowing skin [title] Drink up. [step] Aim to get 6 to 8 glasses of water a day. The water will clear your skin and make it glow because it makes it easier for your body to flush out toxins quickly.\n\nA. Fruit juices or energy drinks will also help flush out toxins. You can also take a 12 ounce glass of mineral spirits and drink it with tea and pills; it is thought to reduce inflammation and other body irritations.\nB. Get yourself a cup of water and add a splash of lemon juice to the water. Drink this up as fast as you can.\nC. [substeps] Carry a water bottle with you all day to make sure you always have water on hand. Drink herbal tea or other non-sugary beverages to hydrate when you're tired of water.\nD. [substeps] To get a good amount of water, avoid the us water standard, which states that you drink 8 glasses of water everyday. If you are pregnant, do not over-hydrate and do not drink anything outside the 8 glasses of water recommended by your doctor.","prompt":"[header] How to get beautiful, glowing skin [title] Drink up. [step] Aim to get 6 to 8 glasses of water a day. The water will clear your skin and make it glow because it makes it easier for your body to flush out toxins quickly.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6582,"error":null,"output":"C"}
{"question_id":"hellaswag:00004324","item_index":285,"attempt":0,"prompt_hash":"80bd140be2c6a3d77c32f2727a0d5e47afe127f1fb9402e6047e4139dc6f7a79","question":"[header] How to stop your stress from harming your child [title] Watch what you say around your child. [step] Children are constantly learning from their parents. Thus, you need to be very cognizant of everything you say when you're around your child.\n\nA. Therefore, as a parent, it's important to practice mindful listening as a form of \" wisdom. \" try listening closely to your child instead of laughing, crossing your arms, or doing anything to just let your child know you are listening.\nB. Learning doesn't just mean tuning your child into what you say. It means hearing your words aloud several times throughout the day.\nC. You'll want to be as restating what you say as you make a conscious effort to keep your words humble and brief. [substeps] Practicing tone-lowering means minimizing drama that can easily shock a child.\nD. Pay attention to the volume of your voice and your tone as well. Even if your child is too young to understand what you mean, they can still understand when you are angry by the volume and tone of your voice.","prompt":"[header] How to stop your stress from harming your child [title] Watch what you say around your child. [step] Children are constantly learning from their parents. Thus, you need to be very cognizant of everything you say when you're around your child.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6583,"error":null,"output":"D"}
{"question_id":"hellaswag:00005750","item_index":286,"attempt":0,"prompt_hash":"99555fa4dd5ff876d82b62be5dd7883a8e28c605293c5a38128e75fdd8a0f1c0","question":"[header] How to get rid of teenage acne [title] Keep hair clean. [step] This step is especially important for teens with long hair. Oily hair or hair products in constant contact with the face can help clog pores.\n\nA. Plus, keeping your hair clean can lower your risk of this type of acne. Shampoo every other day to keep your hair healthy and moisturized.\nB. If you have their face cleanser, shampoo, conditioner, and other moisture-collecting products, reduce contact time and don't wear them all the time. Your scalp needs to have some moisture on a regular basis.\nC. Even those with short hair can see blemishes around the hairline due to oily hair or hair products. Wash your hair regularly and try to keep your hair off of your face as much as possible.\nD. If you have long hair, clean it well and let it grow out naturally. [substeps] Your hair often needs to be shampooed in the shower, especially if you smell, age, or have hair that gets dry and greasy.","prompt":"[header] How to get rid of teenage acne [title] Keep hair clean. [step] This step is especially important for teens with long hair. Oily hair or hair products in constant contact with the face can help clog pores.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6583,"error":null,"output":"C"}
{"question_id":"hellaswag:00000604","item_index":287,"attempt":0,"prompt_hash":"1e30da082238daaeb3efd3053686967fd40210c589d3fe44a216aefe1ec7c639","question":"A black screen is pictured and white words begin to bounce down on the screen. after\n\nA. , the perber machine wakes up and the black is seen and white is shown across the screen.\nB. , a group of young men are pictured playing indoor soccer.\nC. they are done bouncing, the man in blue returns and bowls it back, which continues to bounce on the table one at a time.\nD. , a group with short hair and an under armour appear and begin getting ready mentally for the exercise.","prompt":"A black screen is pictured and white words begin to bounce down on the screen. after","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6581,"error":null,"output":"B"}
{"question_id":"hellaswag:00006812","item_index":288,"attempt":0,"prompt_hash":"57b031a8c1a2dd2ce413dba688a438d54bb1ef85f32a9fbf8038801b98454b0f","question":"[header] How to unlock a door [title] Use a credit card on a spring lock. [step] Though it won't work for deadbolts, it will work for spring locks (the kind that's slanted and a part of a handle). Grab one that's cheap and you don't care about--if it breaks, no harm, no foul.\n\nA. A bendy, laminated card works best. [substeps] Force it between the lock and the frame, flush against the frame.\nB. [title] Push your finger down on the latch to open the door. [step] If you have a mini spring lock, push your finger down through the top of the lock to open it.\nC. [title] Jump into the lock and press the in button. [step] Do this for a quick, forceful push, which the lock will reach and swing open once unlocked.\nD. Once you do that, you can rip the lock right out of the door. If you do not have a credit card, borrow from a friend, or rent one from a local hardware store.","prompt":"[header] How to unlock a door [title] Use a credit card on a spring lock. [step] Though it won't work for deadbolts, it will work for spring locks (the kind that's slanted and a part of a handle). Grab one that's cheap and you don't care about--if it breaks, no harm, no foul.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6670,"error":null,"output":"B"}
{"question_id":"hellaswag:00002587","item_index":289,"attempt":0,"prompt_hash":"4f1fe645b4ea0ebb1566c934a710bec71ad0bbcffcc7e6cec779ec96168e9878","question":"A video of scissor kick jumping at track is shown. A few people take turns at varying heights and all are successful. others\n\nA. act as the ref by hopping on or off the ropes.\nB. are running for other events in the background.\nC. are tested and they do this on the long jump with more success.\nD. are only successful on a very high jump.","prompt":"A video of scissor kick jumping at track is shown. A few people take turns at varying heights and all are successful. others","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6674,"error":null,"output":"B"}
{"question_id":"hellaswag:00003166","item_index":290,"attempt":0,"prompt_hash":"77010755debddb549a628933571d9b56e057807975cdec844e9c7b37a772d0fc","question":"A man lights a piece of paper on fire. he\n\nA. then places it on a stack of sticks.\nB. blows smoke into numerous cars driving by.\nC. begins to smoke something in a bucket.\nD. holds his hand over the flame and sparks fly.","prompt":"A man lights a piece of paper on fire. he","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6673,"error":null,"output":"D"}
{"question_id":"hellaswag:00007496","item_index":291,"attempt":0,"prompt_hash":"9a5a4aa957c19e32ebc0f4bc60b05b9100eebfacad16b7aab86405e910cd4b6b","question":"[header] How to make nougat [title] Line the baking pan. [step] Line the bottom and sides of an 8-inch by 8-inch (20-cm by 20-cm) baking pan with parchment paper. Set it aside for later use.\n\nA. [substeps] If you'd like to chill the bread without parchment paper, pour 2 tablespoons (45 ml) of sugar into a bowl and put it in the freezer to chill. Or, you could refrigerate the mixture for at least one day.\nB. [title] Bring the boiling water to a boil. [step] Over the stove, turn the heat to high to bring the water to a boil.\nC. You will be pouring your maple syrup into the pan after you bake your honey and sugar. [title] Heat the water in the saucepan.\nD. [substeps] Alternatively, you could grease the bottom and sides of the baking pan with butter, shortening, or nonstick cooking spray. Using parchment paper will make the pan easier to clean, though.","prompt":"[header] How to make nougat [title] Line the baking pan. [step] Line the bottom and sides of an 8-inch by 8-inch (20-cm by 20-cm) baking pan with parchment paper. Set it aside for later use.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6668,"error":null,"output":"D"}
{"question_id":"hellaswag:00002851","item_index":292,"attempt":0,"prompt_hash":"68c64d7ce3381abfa19def93a42b2b71e6b57497aed63eea74558479ed7bdeb8","question":"The woman is with the black dog and is styling the dog's hair. Woman shows their diplomas and awards and is with a white puddle combing her hair. woman\n\nA. comb the black dog's hair and plays with it.\nB. is with a journalist on a qlon tv and is painting on her hands and the amazement of a logo is shown.\nC. is hairspray and products present to a client.\nD. talks to the camera and puts her hair up in a clipper, continues combing her hair and talks to the camera.","prompt":"The woman is with the black dog and is styling the dog's hair. Woman shows their diplomas and awards and is with a white puddle combing her hair. woman","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6675,"error":null,"output":"A"}
{"question_id":"hellaswag:00000230","item_index":293,"attempt":0,"prompt_hash":"758e97d38d6a6435f0da1150029f1cc34420c1677b7cc41a5409b57dee846ed5","question":"A woman named julie artzt becker is holding her violin is sitting and talking in front of a blue screen. The woman puts the violin up to her chin and continues to talk. the woman\n\nA. picks up her violin and continues to talk before she starts to play it and stops.\nB. then puts her fingers onto the strings and with her other hand she touches and points on her fingers on the strings as well as taking the bow to her strings every now and then.\nC. finishes the violin and grabs a violin from underneath her.\nD. continues to talk while showing her hands in front of her.","prompt":"A woman named julie artzt becker is holding her violin is sitting and talking in front of a blue screen. The woman puts the violin up to her chin and continues to talk. the woman","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6674,"error":null,"output":"A"}
{"question_id":"hellaswag:00006021","item_index":294,"attempt":0,"prompt_hash":"0d54373ff6a55ad66989679d649ec7416fc66907429adaa55d55fe3250580230","question":"[header] How to rotate in volleyball [title] Know the six court positions. [step] Each team's side of the volleyball court will be filled with two rows of three players each, making for a total of six spots. Though players rotate clockwise, the positions are labeled counter-clockwise.\n\nA. People in each row may move between both sides of the court. [title] Choose a middle spot for dribbling.\nB. The team's red goal is to view the dolphins hidden from the net for defense. Their yellow goal is to view the dolphins hidden from the net.\nC. Here they are : [substeps] Position 1: the right back, where the serving player is. Position 2: the right front, just in front of the right back.\nD. For example, the front court handles five of the players at attention, while the back court handles seven. [substeps] The ball, opening at the end of the rotation, remains between the forward ankle and inward knee.","prompt":"[header] How to rotate in volleyball [title] Know the six court positions. [step] Each team's side of the volleyball court will be filled with two rows of three players each, making for a total of six spots. Though players rotate clockwise, the positions are labeled counter-clockwise.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6668,"error":null,"output":"C"}
{"question_id":"hellaswag:00008610","item_index":295,"attempt":0,"prompt_hash":"ada637d12364e357a8e59e19e6b80e5f0b0d6892236d69018a385214533a966b","question":"[header] How to avoid common problems when installing laminate flooring [title] Be sure you are up to the challenge. [step] Installing new floors is a big task and is fundamental to your home. To be successful, you need to have patience, adequate knowledge, and proper tools.\n\nA. If this doesn't happen, talk to your general contractor, or hire an outside contractor to fix the flooring for you. If you're not sure which to consider, these are the types of questions you should ask yourself.\nB. Learning the basics is your key to success at your job. [substeps] Look for areas of the home that need improvement, such as the basement or master bedroom.\nC. If you can't pull it off the first time, don't believe it. If you don't succeed, don't keep on trying.\nD. Understand the time, energy, and commitment that is required before starting major projects. [substeps] If you choose to hire a professional, be sure they are licensed and experienced.","prompt":"[header] How to avoid common problems when installing laminate flooring [title] Be sure you are up to the challenge. [step] Installing new floors is a big task and is fundamental to your home. To be successful, you need to have patience, adequate knowledge, and proper tools.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6667,"error":null,"output":"D"}
{"question_id":"hellaswag:00005178","item_index":296,"attempt":0,"prompt_hash":"783b3acbbd00af3c1b0ba414cfe978eb84c2c062310b37b2c829bab6d0758c09","question":"[header] How to solder pipe [title] Assemble the tools you'll need for soldering the pipe. [step] Most importantly, you'll need a propane torch. This is a small propane tank with a regulator fitted on top; some have an automatic igniter, while others will require a striker to ignite the gas.\n\nA. The propane torch is also called a pilot-welder as it will allow you to perform the soldering process with a more intense flame. The pilot-welder is safe for use on pipe pipes, but a pilot is better.\nB. You will also need lead-free solder (leaded solder can leach into your water supply, causing health problems). A tube cutter is the ideal tool for cutting copper pipes, but a hacksaw will also work.\nC. Here's the most basic tools you'll need to solder a pipe. [substeps] You can fill a small tank's opening with the following: a amp-3 balance a striker to ignite water [title] Assemble the parts and tools you'll need.\nD. [substeps] If you're working with propane, you'll need torch gaskets which put a locking ring around the fuel source. It doesn't matter which one there is--you just need one.","prompt":"[header] How to solder pipe [title] Assemble the tools you'll need for soldering the pipe. [step] Most importantly, you'll need a propane torch. This is a small propane tank with a regulator fitted on top; some have an automatic igniter, while others will require a striker to ignite the gas.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6669,"error":null,"output":"B"}
{"question_id":"hellaswag:00001608","item_index":297,"attempt":0,"prompt_hash":"36499e8fb54f6c599c82ee098a6dc8d61d3d48a28497f9fc5e6bc13a7ad36065","question":"Brazil finds itself down even more on another argentinian goal on a beautiful pass and shot across the field. to make matters worse\n\nA. , argentina scores on a deep uncontested shot after that.\nB. , several people from the city are shown warming up for the game and are dancing in each other x's.\nC. , the refs have added a substitute slide into their video.\nD. , one of the players is wearing a red hat and has a basket.","prompt":"Brazil finds itself down even more on another argentinian goal on a beautiful pass and shot across the field. to make matters worse","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6670,"error":null,"output":"A"}
{"question_id":"hellaswag:00005059","item_index":298,"attempt":0,"prompt_hash":"86e9bdf8144310c086471c66cd5936d202db55cc8eae460e02c958940739ef7f","question":"[header] How to hide or fix a shaved off eyebrow [title] Choose an eyebrow pencil and powder that is similar to your natural coloring. [step] If you still have one of your eyebrows, this might be a little easier--just match the pencil to the shade of your eyebrow hair. Otherwise, use your hair color as a guide.\n\nA. [substeps] If you have very gray or black eyebrows, choose an eyebrow pencil that is between 1/4 and 1/2-inch (0.64 to 1.27-centimeter) long. [title] Use powder or shaving cream to conceal your brows.\nB. You can even wait for this to be complete--most cuts don't last as long as lashes or false lashes. [title] Take the pencil and powder (roughly 2/3 of which will go into your hair) and get ready to apply.\nC. If you are a brunette or a redhead, look for a color that is one shade lighter than your hair. If you are a blonde, try one shade darker.\nD. [substeps] For any blended color, use a black eyebrow pencil to blend it in. This shade doesn't pull out the full color of the eyebrow, but it leaves just enough to designate it as the back side of your eyebrow.","prompt":"[header] How to hide or fix a shaved off eyebrow [title] Choose an eyebrow pencil and powder that is similar to your natural coloring. [step] If you still have one of your eyebrows, this might be a little easier--just match the pencil to the shade of your eyebrow hair. Otherwise, use your hair color as a guide.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6667,"error":null,"output":"C"}
{"question_id":"hellaswag:00001759","item_index":299,"attempt":0,"prompt_hash":"aab2090826edf52b5b82d0c8fa618eeaecbeb62b43fa72c91981941cb55fc1a0","question":"A man with a blue shirt, blue sweat pants and green socks is standing outside in a large field walking into a circle of cement. Once the guy is on the circle, he begins walking around the circle to find his perfect spot. everything\n\nA. starts to look very interesting as he begins to bump into the wall and fall into it.\nB. is as she was gone, she was standing then spinning around a few times.\nC. is in place as he grabs the end of the hopscotch stick with his foot and uses it to move towards the outline of the jump.\nD. is perfect and then he throws the discus that was in his hand into the field as he is spinning around.","prompt":"A man with a blue shirt, blue sweat pants and green socks is standing outside in a large field walking into a circle of cement. Once the guy is on the circle, he begins walking around the circle to find his perfect spot. everything","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6671,"error":null,"output":"D"}
{"question_id":"hellaswag:00007248","item_index":300,"attempt":0,"prompt_hash":"87bfe6dd41a649bdf00e5f63769228239f0558375b7c914c468e1743c15baf31","question":"[header] How to pamper yourself at home [title] Run yourself a hot bath. [step] Check the temperature to make sure that it isn't too hot. Add some rose or lavender oil to the bath to soothe your mind and relax.\n\nA. Lather your hair with the warm water. Start by tensing your muscles, holding a deep breath.\nB. Then, put your soaking clothes and relaxing medicine into the warm bath. Heat the water to the right temperature before you put your hand in and take a few deep breaths.\nC. Turn on some music and soak in it for a few minutes. [title] Draw or edit pictures of yourself online.\nD. You could also use some lavender scented bubble bath. Dim the lights in your bathroom so it's darker or just go by candlelight.","prompt":"[header] How to pamper yourself at home [title] Run yourself a hot bath. [step] Check the temperature to make sure that it isn't too hot. Add some rose or lavender oil to the bath to soothe your mind and relax.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6672,"error":null,"output":"D"}
{"question_id":"hellaswag:00008228","item_index":301,"attempt":0,"prompt_hash":"e68c49f56439e42ac66b1b5a715b0d449cb7025c7e15c781804f29c0e025fac7","question":"[header] How to spot a fake gucci belt [title] Check the color and logo on the gift bag. [step] All authentic gucci belts come in a gift bag. The bag should be a dark brown color with the double g logo (one backwards capital g facing another capital g) printed all over the surface of the bag, excluding the bottom of the bag's interior.\n\nA. It should also have the name \" gucci \" written on the outside. If you cannot find a gucci belt with the actual gucci name, find a garment store that sells gucci belts.\nB. The silver logo appears under the belt, so it must be fake. [substeps] Regular gucci belts with silver buckled buckles look fake, even if the clothes, shoes, hands, and accessories are genuine.\nC. [substeps] There should also be a dark brown string on top that can be tied to keep the product from falling out. [title] Check the dust bag for the brand name in gold letters.\nD. [substeps] Almost always gucci belts have return stickers on them with the recipient's name, so visit a print shop to get a confirmation of authenticity. [title] Check with your dress code if an actual gucci belt is provided.","prompt":"[header] How to spot a fake gucci belt [title] Check the color and logo on the gift bag. [step] All authentic gucci belts come in a gift bag. The bag should be a dark brown color with the double g logo (one backwards capital g facing another capital g) printed all over the surface of the bag, excluding the bottom of the bag's interior.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6667,"error":null,"output":"C"}
{"question_id":"hellaswag:00001366","item_index":302,"attempt":0,"prompt_hash":"c205c0c591cf1ef4a202467169d3751e718eb10c94df3d57f1ae98da21406bdf","question":"Pole vault is a sport where you run with a pole and stop and swing your body. a few women\n\nA. go down a rock and stop.\nB. watch from around a pole vault.\nC. are in the wading pool.\nD. are in the gym practicing to do this pole vault, trying to do their best.","prompt":"Pole vault is a sport where you run with a pole and stop and swing your body. a few women","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6671,"error":null,"output":"D"}
{"question_id":"hellaswag:00007694","item_index":303,"attempt":0,"prompt_hash":"4e08843450dbca5301bd18467b11271738eb3b110de8f2cbb41e2cbf0e297824","question":"[header] How to cut a limb from a tree [title] Wait until your tree is dormant in the winter to make your cuts. [step] Pruning your tree when it's dormant gives it time to heal before threatening pests arrive. If you have particularly cold winters, wait until the coldest part of winter is over before you prune your tree.\n\nA. However, cutting your tree before the last light frost sets in can set the stage for unexpected pests. [substeps] While it may be tempting to prune your tree until it's freezing, try not to become obsessed with the idea of waiting until the time.\nB. [substeps] Don't cut too soon after temperatures drop below a three-month mark. Let it heal in two weeks or so.\nC. [title] Cut the branches back to their normal length. [step] Depending on your tree's type, when the tree is dormant it may be of no use to you.\nD. [substeps] Pruning in the winter will also help your tree grow more during the spring. Avoid pruning your tree during the growing season or you could stunt its growth.","prompt":"[header] How to cut a limb from a tree [title] Wait until your tree is dormant in the winter to make your cuts. [step] Pruning your tree when it's dormant gives it time to heal before threatening pests arrive. If you have particularly cold winters, wait until the coldest part of winter is over before you prune your tree.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6666,"error":null,"output":"D"}
{"question_id":"hellaswag:00002137","item_index":304,"attempt":0,"prompt_hash":"fbd0ac7fb629c9b076c5d4f74fbf087461939354ec0307f5eac9ff1f07bd6b09","question":"They being their match, one in a white tank top and green shorts, the other in an orange shirt and white shorts. they\n\nA. are both throwing the bats in the air.\nB. are still caught at the marker holding onto the bumper cars.\nC. hit back and forth for some time, the enjoyment of the game seeming more important the winning, and they slap hands at the conclusion.\nD. both throw a shot put at each other.","prompt":"They being their match, one in a white tank top and green shorts, the other in an orange shirt and white shorts. they","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6670,"error":null,"output":"C"}
{"question_id":"hellaswag:00006330","item_index":305,"attempt":0,"prompt_hash":"ef8c261e40f4e670d5cc11bc7d5bd54840002843f1547a245bdd039c073f4dff","question":"[header] How to travel to phuket [title] Travel to phuket with just your passport if you are staying up to 30 days. [step] If you live in western countries and the middle east, you likely do not need a travel visa to visit phuket. If you do not have a passport, visit your country's international travel department.\n\nA. [substeps] It may take a few days for your visa to be approved for processing. If you do not have a passport, you will receive a letter in phuket from the u.s.\nB. Typically, the process includes completing the form, submitting your required documents, getting a passport photo, paying the fees, and submitting everything. [substeps] Visit https: // www.phuket.\nC. [substeps] If you are staying in another country where you do not need a visa, you can apply for a visa in your country. You may apply online at phuket international travel supplies website.\nD. These departments will give you additional information, including the following : [substeps] Travel itinerary if you live in the united states. The secretary of state may also provide you with an itinerary.","prompt":"[header] How to travel to phuket [title] Travel to phuket with just your passport if you are staying up to 30 days. [step] If you live in western countries and the middle east, you likely do not need a travel visa to visit phuket. If you do not have a passport, visit your country's international travel department.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6670,"error":null,"output":"C"}
{"question_id":"hellaswag:00008454","item_index":306,"attempt":0,"prompt_hash":"096ca9927995df0d43ec7f94faefbbde38cf6919d0a4eac9a0940b6aa39a0d6b","question":"[header] How to make an almond milkshake [title] Pour the almond milk into a blender. [step] You can choose plain, vanilla, or chocolate depending on your flavor preference. For a healthier shake, choose unsweetened almond milk and let the other ingredients add the sweetness.\n\nA. [title] Pulse the blender with a rolling pin or cheese grinder at medium speed for about 1 minute. [step] This gives the almond milk time to break down the sugars and other important calories.\nB. Keep blending the ingredients until the almond milk is frothy. [substeps] To increase the strength of the shake, go easy with the unsweetened almond milk and shake the liquid completely.\nC. [substeps] If you want to create an almond milk smoothie, rather than a shake, add 8 oz. ( 237 ml) of non-fat yogurt.\nD. However, to get that butterscotch syrup, you'll need to be sweet. If you prefer a more intense shake, you can also simply top it with almond milk instead.","prompt":"[header] How to make an almond milkshake [title] Pour the almond milk into a blender. [step] You can choose plain, vanilla, or chocolate depending on your flavor preference. For a healthier shake, choose unsweetened almond milk and let the other ingredients add the sweetness.","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6668,"error":null,"output":"A"}
{"question_id":"hellaswag:00002549","item_index":307,"attempt":0,"prompt_hash":"75abd7cd9614f3e627062b1eab3af4afdfbf9c5999b81e2af9c4e5f58de6e499","question":"A man us shown running on a track with a red surface. he\n\nA. runs with the blue line on the right and a box measuring his distance.\nB. runs full out, jumping and throwing obstacles as he goes.\nC. starts running backwards into the sand in front of a large pile of sand.\nD. then runs full speed and jumps a tall high jump beam.","prompt":"A man us shown running on a track with a red surface. he","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6670,"error":null,"output":"C"}
{"question_id":"hellaswag:00007116","item_index":308,"attempt":0,"prompt_hash":"ceb37aaf615f717dd97d198105a221b18d91a7792b8192ef37a141983b620d98","question":"[header] How to find unclaimed money [title] Collect the names and past and current addresses for yourself and/or any others for who you want to search for unclaimed property. [title] Go to www.missingmoney.com, showmemymoney.com/or www. Myunclaimed.com enter your name and hit search.\n\nA. [title] Contact the application and let it search to find unclaimed property. [step] You will be taken to a web page that suggests you have entered the right name, address and previous contact information.\nB. [title] Take the name and current address and enter it on the search field at the top of your search results. [title] Click on \" start my search \".\nC. [title] Look for friends or coworkers that earn unclaimed money. [step] They will have fake names to provide potential buyers with time to find them.\nD. [title] Review the list of names to see if your name is on the list. [step] If your name is not anywhere on the list skip to step 8.","prompt":"[header] How to find unclaimed money [title] Collect the names and past and current addresses for yourself and/or any others for who you want to search for unclaimed property. [title] Go to www.missingmoney.com, showmemymoney.com/or www. Myunclaimed.com enter your name and hit search.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6669,"error":null,"output":"D"}
{"question_id":"hellaswag:00005520","item_index":309,"attempt":0,"prompt_hash":"d1aad406fc7f27c70ec6b14ca14c40fd9c57a404b5e493fd3a36d24aeeb5c846","question":"[header] How to love your body after having a baby [title] Avoid comparing your body to others. [step] Try not to compare your post-baby body to your friend who can somehow fit into a size 2 after giving birth or to celebrities who are photographed in bikinis after they have a baby. Giving birth affects every woman's body in different ways, and comparing yourself to others will only lead to unrealistic expectations and negative thoughts.\n\nA. Focus instead on your body and your needs, rather than the post-baby bodies of others. [substeps] Start by blocking out images from the media that show \" perfect \" post-baby bodies.\nB. [substeps] Realize that you are an individual who's choosing your body. Your body is a compliment to your body as it is.\nC. [substeps] This applies to both male and female bodies and teens. Modesty seems to fit more honestly when you're surrounded by people who are also other people.\nD. Put aside the critical evaluations, weekly check ups and talk to your partner about your body. If your body looks big to you, remember that their body is bigger.","prompt":"[header] How to love your body after having a baby [title] Avoid comparing your body to others. [step] Try not to compare your post-baby body to your friend who can somehow fit into a size 2 after giving birth or to celebrities who are photographed in bikinis after they have a baby. Giving birth affects every woman's body in different ways, and comparing yourself to others will only lead to unrealistic expectations and negative thoughts.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6671,"error":null,"output":"A"}
{"question_id":"hellaswag:00004042","item_index":310,"attempt":0,"prompt_hash":"389402e8ffd92b63a83b49f68724615e41035db1b1083aed0c8e33f34a2b1d75","question":"[header] How to choose a flattering plus size swimsuit [title] Find a style that fits you well. [step] Nothing will flatter you if you feel uncomfortable or the swimsuit sags on you and fails to make the most of your figure. Remind yourself while shopping for a swimsuit that you deserve something of quality that you feel good in; don't settle for less.\n\nA. [title] Look for styles made for curvier figures. [step] For example, the 50s style one-piece swimsuits were made to accentuate and celebrate curves and are very flattering.\nB. [substeps] Versatility is always more important with a swimsuit that fits you well. Provided you look great in your swimsuit (and you have great leg work), a good figure will almost instantly make anyone's day easier.\nC. [substeps] Before you begin trying on a swimsuit to see if it's right for you, test a few different bottoms around the beach to find the right fit. Remember that height is a critical factor of swimsuit length.\nD. [title] Find a color that compliments your body and will really scream masculine. [step] If you're having trouble choosing a swimsuit or a drawing that doesn't show all your natural features, stay away from gingham or sunless fabrics.","prompt":"[header] How to choose a flattering plus size swimsuit [title] Find a style that fits you well. [step] Nothing will flatter you if you feel uncomfortable or the swimsuit sags on you and fails to make the most of your figure. Remind yourself while shopping for a swimsuit that you deserve something of quality that you feel good in; don't settle for less.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6958,"error":null,"output":"A"}
{"question_id":"hellaswag:00007713","item_index":311,"attempt":0,"prompt_hash":"0e69b9552522a2fc480fc9fbcf2ff1d130e3d3530d18e735a2580b5ed2ef4f6a","question":"[header] How to renew your wedding vows [title] Know when the time is right. [step] Any married couple can renew their vows any time they want, but having a reason for doing so can make the occasion feel more special. Maybe you got married in a courthouse and never got to have your loved ones there to witness your big day.\n\nA. Maybe you couldn't afford a big wedding the first time around but are ready to throw a huge gala now. Maybe you or your partner have overcome a major life difficulty, such as an illness or a career crisis, and you want to show that your love is stronger than ever.\nB. Or maybe you got married in a historic city and your fiancé-to-be is marrying your best friend. Whatever the case is, it has to be right before your big day and there is absolutely no wrong way to do it-the choice is yours alone.\nC. Maybe you married in the park and saw your future nephew or cousin light up the place. Whatever the case, there will be a later date as you get to discuss when the time is right to renew your vows with your spouse.\nD. Maybe you've just been bought the biggest wedding gift ever! Just make sure you're not delaying or surprising your loved ones. [substeps] Be sure to remind your friends and family of your commitment to the future during your ceremony.","prompt":"[header] How to renew your wedding vows [title] Know when the time is right. [step] Any married couple can renew their vows any time they want, but having a reason for doing so can make the occasion feel more special. Maybe you got married in a courthouse and never got to have your loved ones there to witness your big day.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6958,"error":null,"output":"A"}
{"question_id":"hellaswag:00009069","item_index":312,"attempt":0,"prompt_hash":"26c27ae55f0b038e156a5be70d4b93075595bf9fe27a3dff56543bd7c73d3fc3","question":"[header] How to get out of awkward situations [title] Wait for a lull in the conversation. [step] Conversations naturally have times where they drop off or have a lull in them. Typically this is when a new topic starts if you want to keep the conversation going, but it's also the perfect time to take the steps to end the conversation.\n\nA. [substeps] Respect when a new topic rolls up. Oftentimes it simply goes from \" hey, i just missed some delicious chicken with buns on it, \" to \" awesome, see you around \" or \" let's get out there and have some food.\nB. Use transition words like so, anyway, or well to begin to say goodbye. [title] Summarize key points of the conversation.\nC. If you see a lull in someone's conversation, you can use it to your advantage. Maybe you can say something like, \" hey john, you seem to be a little standoffish lately.\nD. [substeps] A pause in a conversation indicates that you're ready to say more. For example, your new friend might say, \" so you're not doing well today.","prompt":"[header] How to get out of awkward situations [title] Wait for a lull in the conversation. [step] Conversations naturally have times where they drop off or have a lull in them. Typically this is when a new topic starts if you want to keep the conversation going, but it's also the perfect time to take the steps to end the conversation.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6958,"error":null,"output":"C"}
{"question_id":"hellaswag:00005404","item_index":313,"attempt":0,"prompt_hash":"869d592ba0b198abb12b383f417dab8a1f258f0acd5c9233f900feb8d4f80ec1","question":"[header] How to wear an ear warmer [title] Consider the color. [step] Your ear warmer should be a fashionable accessory that works with the rest of your clothes. When choosing an ear warmer to wear, select one that will match the colors you're wearing.\n\nA. Black and brown ear warmers also work well, but a zebra ear warmer will have more space, and not provide as much space as beige or brown ear warmers. If you wear the ear warmer too often, you may find it hard to wear it with turquoise, teal, and bronze tones.\nB. Choosing a brightly colored ear warmer can cut down on the price since you can also find ear warmers at thrift stores. [substeps] Ear warmers tend to be sticky to the touch, making them a good option for warm clothes.\nC. You can wear your ear warmer in a variety of colors, or, you can choose a more eye-catching color that works well with the rest of your wardrobe. Different ear warmer shades will match more colors than the rest of your wardrobe.\nD. While picking the same colors works, it's not the only way to coordinate your colors. Make sure that your ear warmer and clothes stay within the same family of tones.","prompt":"[header] How to wear an ear warmer [title] Consider the color. [step] Your ear warmer should be a fashionable accessory that works with the rest of your clothes. When choosing an ear warmer to wear, select one that will match the colors you're wearing.","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6957,"error":null,"output":"C"}
{"question_id":"hellaswag:00000891","item_index":314,"attempt":0,"prompt_hash":"02500d5c2150dec0fbf0c9d88911ce9e6563e79823141c547796c250694f3340","question":"They take another drink and dump it out in the sink. They pick up the towel and wipe their face. they\n\nA. push the trashcan next to them to celebrate.\nB. grab a cup and turn the sink on.\nC. then get their camel whistles ready.\nD. grab makeup brushes and hold them up.","prompt":"They take another drink and dump it out in the sink. They pick up the towel and wipe their face. they","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6956,"error":null,"output":"B"}
{"question_id":"hellaswag:00008378","item_index":315,"attempt":0,"prompt_hash":"92b247e67a08840b810ee965fbe57193ef3873d05fb3b223fa56ca231761f59d","question":"[header] How to get a temporary job over 50 [title] Keep in mind the reason you are looking for a temporary job. [step] Are you doing it for : [substeps] Money new skills social networking fun [title] Think outside the box. [substeps] Don't limit yourself to the type of work you have always done.\n\nA. In fact, if you are more ambitious and want to break into the industry, you may have more in common with other people even better in your field. [title] Determine what college you want to go to.\nB. If you do this, people will take advantage of you! [title] Gauge potential employers based on what they know about you. [substeps] Offer your work to firms who might be interested in you.\nC. Consider taking a job doing something that you always thought you might like to do. Temporary jobs are a great way to try out new types of work without making long-term commitments.\nD. It can be hard for you to concentrate, and sending out long-winded messages on social media could create a void. Try to volunteer in work you love or that is not the return of your resume.","prompt":"[header] How to get a temporary job over 50 [title] Keep in mind the reason you are looking for a temporary job. [step] Are you doing it for : [substeps] Money new skills social networking fun [title] Think outside the box. [substeps] Don't limit yourself to the type of work you have always done.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6724,"error":null,"output":"C"}
{"question_id":"hellaswag:00005920","item_index":316,"attempt":0,"prompt_hash":"e0f38232477ee7cf0f208d4b8271654ec81d2afbf9d966a719cc3f260920d832","question":"[header] How to make a mayonnaise cake [title] Using the box directions, preheat the oven to the degrees specified on the box, usually this is 350f or so. [step] Using wire whisk or spatula, create a well in the center of the cake mix. [title] Lightly spray pam in the baking dish and be careful not to spray too high on the sides.\n\nA. [title] Place mayonnaise in the middle, the scones will go first. [title] Use line a, layer cake topping onto scones.\nB. [title] Empty contents of cake mix into a large bowl. [step] If you prefer, you can empty the bowl of cake mix into a food processor and then add the mayonnaise.\nC. [title] Now you will need to lay the cake cake on a cookie sheet with a crust so it starts to soften. [title] Once the cake is baked on you will probably want to coat with sugar, cream, to season.\nD. [title] Stir eggs one at a time until smooth. [title] Dip a pastry brush in the mixture and brush the crust using zig zagged lines.","prompt":"[header] How to make a mayonnaise cake [title] Using the box directions, preheat the oven to the degrees specified on the box, usually this is 350f or so. [step] Using wire whisk or spatula, create a well in the center of the cake mix. [title] Lightly spray pam in the baking dish and be careful not to spray too high on the sides.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6724,"error":null,"output":"B"}
{"question_id":"hellaswag:00005926","item_index":317,"attempt":0,"prompt_hash":"3e63c6f6cc32fa9f7d79ac0162455f5eb0434d678cc234995df51db2e6507111","question":"[header] How to make a pin [title] Find a small, square-shaped image that you like. [step] Aim for something that is about 1 inch by 1 inch (2.54 centimeters by 2.54 centimeters). If necessary, resize the image using an image editing program or a photocopier.\n\nA. If you do not have an image editing program, photoshop companies will allow you to get an image that matches the object. [title] Cut out a sheet of color paper to the size you want your pin to be.\nB. [title] Cut the image out, but leave a ½ inch (1.27 centimeters) border all around it. [step] You will need this excess material to wrap around the back of your pin.\nC. [substeps] The photo will be 7-10 inches x 7-10 inches. If you plan on colored photos, you can make your pin yourself or you can find images online that match.\nD. [title] Cut out one of the 2 blank sides of a photo. [step] The 2 first pictures will be vertical and two horizontal, spaced five to six inches on either side.","prompt":"[header] How to make a pin [title] Find a small, square-shaped image that you like. [step] Aim for something that is about 1 inch by 1 inch (2.54 centimeters by 2.54 centimeters). If necessary, resize the image using an image editing program or a photocopier.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6724,"error":null,"output":"B"}
{"question_id":"hellaswag:00002444","item_index":318,"attempt":0,"prompt_hash":"518eea78120b1e08b8ca8e9aacf38f8c8523159bf138e90f295ac7bcd1982390","question":"A chef is seen speaking to the camera and begins cutting up ingredients. He cuts up several pieces of food using different knives. he\n\nA. continues cutting up the ingredients and ends by showing off several knives.\nB. polishes the food off and displays it to the camera.\nC. dips up a tomato into water and holds them up to the camera.\nD. puts them into various bowls while speaking into a microphone.","prompt":"A chef is seen speaking to the camera and begins cutting up ingredients. He cuts up several pieces of food using different knives. he","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6723,"error":null,"output":"A"}
{"question_id":"hellaswag:00000097","item_index":319,"attempt":0,"prompt_hash":"75fc5d14ceaced4a6d7879ee12b2b41de498507e31fc5b33c80a1240d8b08041","question":"She picks the knife back up and places it in the appliance and slowly draws it back several times. She slices a tomato on a wood block with the knife. she\n\nA. decorates the knife with green and red maracas.\nB. positions the knife front and back and finishes by pushing it straight across the square board.\nC. then demonstrates how to sharpen the knife and finishes by putting a metal rod in front of the plate of spaghetti that she just sharpened.\nD. grabs a second knife and puts it in the appliance.","prompt":"She picks the knife back up and places it in the appliance and slowly draws it back several times. She slices a tomato on a wood block with the knife. she","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6724,"error":null,"output":"D"}
{"question_id":"hellaswag:00005314","item_index":320,"attempt":0,"prompt_hash":"15f250122030a48a61101e27e3a9cc75b3aa675752c32bddac59458205dedddb","question":"[header] How to cosplay as gumi (panda hero version ) [title] Collect many reference photos. [step] Full body shots will be best, although, there is no official cosplay costume for this. Vocaloid is meant to be interpreted in each fan's own way.\n\nA. Do not fret if you can't find a 100% perfect pair of pants, wig, shirt, etc, because vocaloid is all about being spontaneous, creative and true to yourself! [title] Find a proper wig. [step] In order to do this cosplay, you will need a proper wig.\nB. [title] Collect cut-up pictures of gumi hatcher. [step] These should be picture-size and taller, and generally not quite like gumi hatcher.\nC. Due to this, a full body shot will give in to this hocus-pocus vibe. Just be honest when testing out a plan.\nD. For example, if you're portraying gumi from her photograph from that santa claus scene, you would want to use reverse effects, such as pi and pie pop. No matter what; keep it happy, and photograph yourself looking like gumi at night before bed.","prompt":"[header] How to cosplay as gumi (panda hero version ) [title] Collect many reference photos. [step] Full body shots will be best, although, there is no official cosplay costume for this. Vocaloid is meant to be interpreted in each fan's own way.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5933,"error":null,"output":"A"}
{"question_id":"hellaswag:00002254","item_index":321,"attempt":0,"prompt_hash":"3a0c8a880ba225253c90e0d15384e52cd5b0d54b498e0e701612466f66646b69","question":"A woman is seen running down a road while looking at her watch and pointing to things in front of her. she\n\nA. climbs in a race car and races on the track.\nB. walks around a set of stairs and picks up a big stick to run down a road.\nC. asks to speak to another while standing in a position and begins running down the road.\nD. continues running down the street while other people applaud her and she keeps running.","prompt":"A woman is seen running down a road while looking at her watch and pointing to things in front of her. she","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5947,"error":null,"output":"D"}
{"question_id":"hellaswag:00000981","item_index":322,"attempt":0,"prompt_hash":"bea025a7ace25499f4733dd1f5a85c55a46dd0b807dbd3a45696071529726986","question":"A shot is seen of two men performing an arm wrestling match together while the audience cheers. one man\n\nA. wins by curlew flips and the audience applauds.\nB. beats the other and the two hold up their arms to cheer and everybody celebrates.\nC. holds the other man's other arm while the competitors continue fighting around each other.\nD. is then shown put down and the third man succeeds in the match.","prompt":"A shot is seen of two men performing an arm wrestling match together while the audience cheers. one man","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5947,"error":null,"output":"C"}
{"question_id":"hellaswag:00001713","item_index":323,"attempt":0,"prompt_hash":"2269c12593b3241e84e70c3506ea8ef125d5d4ad4dc806b923c92ffa9e5a9dfb","question":"An overhead view of a blue pool table is shown. a man\n\nA. begins using the glasses to measure the ball.\nB. takes shot with the cue.\nC. is shown in a canoe paddling as he moves it between large rocks.\nD. plays a game of pool in the water.","prompt":"An overhead view of a blue pool table is shown. a man","gold":"B","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5945,"error":null,"output":"D"}
{"question_id":"hellaswag:00007292","item_index":324,"attempt":0,"prompt_hash":"1a6d5437fb5d0b6b891a0566e7ff365a6f82e69cfd0c539e8801996c9764ce40","question":"[header] How to dress like a hogwarts student [title] Select your house. [step] Choose one whose traits you admire, or take an online quiz. Every single student has a house that they belong to at hogwarts, so make sure you choose a house first thing.\n\nA. At the same time, make sure your house is comfortable and doesn't attract \" hogwarts \" with drugs, alcohol, or cigarettes, unless you are a real life person. [substeps] You can also draw on traditional family houses, such as the miniature beehive or baroque houses, to start.\nB. You'll look like a crazy accused of murder in a haunted house, so it's better to be safe than sorry. [substeps] You can choose to live at a hogwarts castle if you'd like.\nC. Each house has its own house colors, which affect the colors of the uniforms. Gryffindor's colors are red and gold, slytherin's are green and silver, and hufflepuff's are black and yellow.\nD. [title] Expect to pay more for your house, especially if you get into trouble in the process. [step] It may seem enormous, but it looks incredibly cool.","prompt":"[header] How to dress like a hogwarts student [title] Select your house. [step] Choose one whose traits you admire, or take an online quiz. Every single student has a house that they belong to at hogwarts, so make sure you choose a house first thing.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5931,"error":null,"output":"C"}
{"question_id":"hellaswag:00009972","item_index":325,"attempt":0,"prompt_hash":"32267505c934b6cfdab781e8d37b7d8328e57b2b1beaf8d9dab1311f7977068c","question":"[header] How to comfort a pregnant teen friend [title] Listen and offer support with open arms. [step] Let your friend know that you are willing to listen if she needs to talk. However, if your friend says she needs some time to process things on her own first, then do not pressure her to talk.\n\nA. Remember that this is your friend's call and does not have to be a formal declaration. [substeps] If your friend tells you, \" i really need to talk, \" then you might say, \" i'd be happy to listen to anything.\nB. Continue being there for her, even if that means checking in regularly, offering support, agreeing on the sofa or going for walks. Showing your care is a great indicator that your friend is thinking about pursuing her dreams and your support cannot be understated.\nC. Give her time and just remind her that you are there for her if she want to talk. [substeps] Try saying something like, \" i know you must be overwhelmed right now, but i am here for you and i am willing to listen if you ever need to talk.\nD. [substeps] Keep your arms close to your body, but do not put them around your friend's abdomen. Let her take it all in, showing that you are there for her.","prompt":"[header] How to comfort a pregnant teen friend [title] Listen and offer support with open arms. [step] Let your friend know that you are willing to listen if she needs to talk. However, if your friend says she needs some time to process things on her own first, then do not pressure her to talk.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5931,"error":null,"output":"C"}
{"question_id":"hellaswag:00001491","item_index":326,"attempt":0,"prompt_hash":"f23805400536993fef99e813a72b52bf3338c2b4713e0fd9bf6b75ab018d5e4a","question":"The man lights a torch. The man uses the torch to light the wood in the fire pit. the man\n\nA. smooths it out and cuts it with the knife.\nB. grabs some sand to build a fire in the pit.\nC. touches a piece of wood with the torch.\nD. sits by the fire pit, enjoying the warmth.","prompt":"The man lights a torch. The man uses the torch to light the wood in the fire pit. the man","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5943,"error":null,"output":"D"}
{"question_id":"hellaswag:00000396","item_index":327,"attempt":0,"prompt_hash":"c9171e4a452f730021d1e81470ea50610e54d7a43cafca1c9539516d6cb1e9e8","question":"Several individuals are dressed in workout clothes, tying up their shoes, and placing numbers on their shirts. all of the people\n\nA. then begin milling around the gym while audience members see their shoes.\nB. are in a parallel gym where they are adjusting their equipment.\nC. are then shown in the gym, sitting, resting, doing weights, cycling, and doing websites with their phone number.\nD. meet at the starting line and begin a marathon throughout the city.","prompt":"Several individuals are dressed in workout clothes, tying up their shoes, and placing numbers on their shirts. all of the people","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5942,"error":null,"output":"D"}
{"question_id":"hellaswag:00003189","item_index":328,"attempt":0,"prompt_hash":"f4635d690a33a16ccd9d9a4e07abc46caef652e4ac18c2077925b241a640339b","question":"A man and a boy scout talk in a kitchen, then the boy scout shows an electric trimmer, a machete and a big knife, and the man shows a rod and a small knife. Then, the boy shows his hand bleeding, after the man wraps the bleeding hand. the adult\n\nA. shows a larger knife with a knob, and the boy and the boy talk inside a bathroom.\nB. excitedly is holding the rod, cut puatre on a green potato on the table and put the potato on the sieve.\nC. trimmer is the only tool for sharpening knives.\nD. shows to the boy scout to trim a wood rod with a knife who again cut a finger, and the man wraps the finger.","prompt":"A man and a boy scout talk in a kitchen, then the boy scout shows an electric trimmer, a machete and a big knife, and the man shows a rod and a small knife. Then, the boy shows his hand bleeding, after the man wraps the bleeding hand. the adult","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5939,"error":null,"output":"A"}
{"question_id":"hellaswag:00003578","item_index":329,"attempt":0,"prompt_hash":"92760806aaa5e8cbfdcc1f4ff122523220d4c8509a6317459fec22719c4200d5","question":"[header] How to get rid of stretch marks fast [title] Drink plenty of water. [step] Water is the quickest and most efficient way to minimize the appearance of stretch marks. Properly hydrated skin is softer and more supple, causing your skin to appear more toned and eliminating some of the \" bumpy \" qualities of stretch marks.\n\nA. [substeps] A good rule of thumb is to drink at least 2.2 liters (7.9 us gal) of water a day. For most people, aim for 106 glasses (or a couple of liter bottles) of water.\nB. Keeping hydrated will also help prevent new stretch marks from forming. [substeps] Specific instructions vary, but consider drinking up to ten glasses of water daily to keep your skin hydrated and minimize the appearance of stretch marks.\nC. [substeps] Drink at least eight 8-ounce glasses of water a day so that your body is ready for the stretch. When you work out, water will help break down the tight, fast-forming muscles, help your stretch easier, and provide moisture.\nD. [substeps] Stay away from sugary drinks, sodas, fluid-filled beverages and sugary snacks. If you want to minimize your stretch marks, try: water glasses, non-alcoholic waters, decaf coffee or tea.","prompt":"[header] How to get rid of stretch marks fast [title] Drink plenty of water. [step] Water is the quickest and most efficient way to minimize the appearance of stretch marks. Properly hydrated skin is softer and more supple, causing your skin to appear more toned and eliminating some of the \" bumpy \" qualities of stretch marks.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5931,"error":null,"output":"B"}
{"question_id":"hellaswag:00005392","item_index":330,"attempt":0,"prompt_hash":"afb91d70dbc71fd02893e2a96676ab11721e3b0a74d7f4e98d8d60f5676141ba","question":"[header] How to stop your toddler from fake crying [title] Ask yourself whether your toddler is fake crying to get attention. [step] Toddlers fake cry for a variety of reasons, but it most often is to get a reaction from their parents. When you understand why they are fake crying, you can tailor your reaction to meet their needs.\n\nA. [substeps] Think about how the child's fake crying is interpreted in the eyes of you. If it is aimed at them, you may want to put an ad in your newspaper to get people to recognize it.\nB. [substeps] Understand that fake crying is often the result of poor self-esteem. Toddlers may be afraid that their mother believes they are a loser, but they may really be faking it to get attention and support.\nC. If you clearly understand the underlying reason for their fake crying, it may be easier to sympathize with them in the future. [substeps] Individuals faking crying may seek attention by calling themselves \" i am the princess \" or \" samantha is a princess.\nD. [substeps] Many toddlers fake cry to get attention if they are competing with other siblings. Some toddlers begin fake crying at the birth of a young sibling or if they are often around infants.","prompt":"[header] How to stop your toddler from fake crying [title] Ask yourself whether your toddler is fake crying to get attention. [step] Toddlers fake cry for a variety of reasons, but it most often is to get a reaction from their parents. When you understand why they are fake crying, you can tailor your reaction to meet their needs.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5934,"error":null,"output":"D"}
{"question_id":"hellaswag:00007680","item_index":331,"attempt":0,"prompt_hash":"c6be4a1951f49c58634ca48979cd88a71c5c6d6e8190ba96f0d22106ce505951","question":"[header] How to bake boneless skinless chicken [title] Preheat the oven to 177 degrees celsius (350 degrees fahrenheit). [title] Remove the chicken breasts from the refrigerator. [step] Make sure to use fresh chicken breasts within two days of placing them in the refrigerator.\n\nA. [title] Combine the grease, ice water, chicken stock, and salt in a small bowl. [step] You can use store-bought marinade if you are having trouble seasoning the chicken.\nB. [substeps] If using frozen chicken, thaw it in the microwave or in a pot of cold water. Frozen chicken can be stored up to nine months.\nC. Don't unwrap the chicken yet. [substeps] The easier you get cooking the chicken, the more fat and sugar will come out of the skin.\nD. [title] Get out the meat tenderloin and side roast on a clean plate. [step] Make sure to keep away from the breasts' bones.","prompt":"[header] How to bake boneless skinless chicken [title] Preheat the oven to 177 degrees celsius (350 degrees fahrenheit). [title] Remove the chicken breasts from the refrigerator. [step] Make sure to use fresh chicken breasts within two days of placing them in the refrigerator.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5935,"error":null,"output":"B"}
{"question_id":"hellaswag:00002859","item_index":332,"attempt":0,"prompt_hash":"d20f13935ff736e867a7f59d6c950b11d36aede20724fe9b59c6f7ed98a454f8","question":"We see the man do a handstand on one hand. The man finishes, grabs his shirt, and walks off. we\n\nA. see video clips in the closing screen.\nB. see the ending screen at.\nC. see frle news clipping on the screen.\nD. see a few names on the screen.","prompt":"We see the man do a handstand on one hand. The man finishes, grabs his shirt, and walks off. we","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5941,"error":null,"output":"D"}
{"question_id":"hellaswag:00003336","item_index":333,"attempt":0,"prompt_hash":"6675bd35297685604442e602c09cd5f5e07c7a9b569f1345b32bdda265e0b496","question":"[header] How to discipline cats [title] Recognize the complications. [step] Cats can be easy or difficult to train. Dogs are more eager to please their owners, which makes them generally easier to train than cats.\n\nA. Because of this, cats respond to rules differently. If you want to discipline your cat but you don't have many resources to teach them, then you should stop your training early.\nB. [substeps] Your cat is more likely to obey if it feels like there are rewards for its behavior. Reward is the sign of a desired behavior, not just a reward.\nC. However, cats can be trained as long as you understand how cats think and interact with their humans. Training a cat takes a lot of patience, especially since it is so hard to motivate him.\nD. It can be far from healthy to train a cat that is getting older and more rambunctious. [substeps] Older cats in their twenties or older can seem more handful-minded, but older cats in their twenties can be more independent and adventurous.","prompt":"[header] How to discipline cats [title] Recognize the complications. [step] Cats can be easy or difficult to train. Dogs are more eager to please their owners, which makes them generally easier to train than cats.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5935,"error":null,"output":"C"}
{"question_id":"hellaswag:00003122","item_index":334,"attempt":0,"prompt_hash":"848eb239d7362d9f3c111d39525553d5321c16ab882392cef5253ade79c00075","question":"A man is lifting weights in a weight room, pulling a large barbell up to his chest. he\n\nA. lifts the barbell over his head three times before dropping it.\nB. stands shaking before lifting it above his head, trying to hold it in place.\nC. drops the barbell onto the ground and he is seen smiling at the camera.\nD. puts his right hand down, and drops it.","prompt":"A man is lifting weights in a weight room, pulling a large barbell up to his chest. he","gold":"B","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5941,"error":null,"output":"A"}
{"question_id":"hellaswag:00003626","item_index":335,"attempt":0,"prompt_hash":"6ff6690f9fb050f95794fe18d0a2a667e76b48f2c3930ead3b7362022dac9666","question":"[header] How to plant a shrub [title] Consider possible locations. [step] It's important to choose a plant that will thrive where you place it. You may not have a location that's right for a shrub you want, but you can evaluate the available space to choose a plant that's perfect for that spot.\n\nA. Before you buy your plant, take time to evaluate what type of shrub will thrive in the location you choose. Make notes of sun exposure, temperature fluctuations, soil moisture, root growth area, and soil composition and acidity.\nB. [substeps] The most popular choice, find a bush where it's about the size of a plant you want and tear it apart. Poke holes in the top, bottom, and sides of the bush.\nC. [substeps] You can plant your shrub anywhere you like, whether it be your backyard or outside. Forget about the weather when you plant it if it's indoors or outdoors.\nD. [substeps] You can find a standard topographical map online or at a gardening store, and you can also ask a store employee what plants grow best in a given spot. If you decide to place your shrub in an area that's ideal for outdoor use, research the location's approximate brilliance before you plant.","prompt":"[header] How to plant a shrub [title] Consider possible locations. [step] It's important to choose a plant that will thrive where you place it. You may not have a location that's right for a shrub you want, but you can evaluate the available space to choose a plant that's perfect for that spot.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5938,"error":null,"output":"A"}
{"question_id":"hellaswag:00002927","item_index":336,"attempt":0,"prompt_hash":"3dbcdf6744a5028d17148e97ec0818d990b4b2c08139a1466681b0ca7d0d346c","question":"A person is skiing down a hill of snow. They pass a group of people snowboarding. they\n\nA. get to the bottom of the hill and stop in front of a small building.\nB. get up the hill again.\nC. pass a home and a group of cars parked in a driveway.\nD. flip side to side down the hill.","prompt":"A person is skiing down a hill of snow. They pass a group of people snowboarding. they","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5941,"error":null,"output":"A"}
{"question_id":"hellaswag:00006761","item_index":337,"attempt":0,"prompt_hash":"855da6f4b6b0040fb646a246974096abddd7423543c428fb518f90ec27800ff0","question":"[header] How to make shrimp cocktail [title] Prepare court bouillon for a tastier shrimp cocktail. [step] You've probably heard of bouillon cubes, right? They're added to hot water to make it more flavorful. Although you certainly don't need to make court bouillon, the addition of wine, herbs, and other seasonings takes this shrimp cocktail dish a couple steps closer to perfection.\n\nA. [title] Add all the ingredients to a large, covered pot and bring to a boil. [step] Don't worry about tying a.\nB. [title] Place the shrimp on a cooling rack covered with a clean towel. [step] Invert the shrimp onto the towel to rest for a few minutes.\nC. You can still prepare a tastier shrimp cocktail at home by making this instead. You just use the ingredients that you've added, as well as herbs, spices, and potatoes.\nD. [title] Grease and prepare an individual bouillon shaker. [step] The ready-to-pour cup of bouillon can be made up from whole or sliced shrimp.","prompt":"[header] How to make shrimp cocktail [title] Prepare court bouillon for a tastier shrimp cocktail. [step] You've probably heard of bouillon cubes, right? They're added to hot water to make it more flavorful. Although you certainly don't need to make court bouillon, the addition of wine, herbs, and other seasonings takes this shrimp cocktail dish a couple steps closer to perfection.","gold":"A","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5938,"error":null,"output":"B"}
{"question_id":"hellaswag:00007222","item_index":338,"attempt":0,"prompt_hash":"7eb6acf39998aa8df73a328c9e82268393f11a23abf43b48b341b846f296dbdf","question":"[header] How to use the chinese birth gender chart for gender selection [title] Find out the mother's lunar age at the time of conception. [step] The chinese use a lunar calendar, which is different from the western gregorian calendar. For this reason, you're going to have to calculate the mother's age.\n\nA. [substeps] Key citations can be found for as many as 35 weeks after you have this chart. For instance, if your mother is 40/60 years old, you would find her 23 months after the month you got the form.\nB. To do this, all you have to do is look up her age on the calendar that is available on modern computers. [substeps] The chinese age chart states that the months are the first years of her pregnancy, which is divided by six.\nC. [substeps] First of all, add one year to however old you are right now. You're 32 years old right now? According to the chinese calendar, you're at least 33 (possibly 34).\nD. That will tell you how many days have passed since conception. If the mother has multiple cycles, both men and women must be conceived.","prompt":"[header] How to use the chinese birth gender chart for gender selection [title] Find out the mother's lunar age at the time of conception. [step] The chinese use a lunar calendar, which is different from the western gregorian calendar. For this reason, you're going to have to calculate the mother's age.","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5938,"error":null,"output":"D"}
{"question_id":"hellaswag:00007727","item_index":339,"attempt":0,"prompt_hash":"a6973fde073b111c02aac88cee8e195d41c752b723cce443b988e34c005642d8","question":"[header] How to use color blocking in your wedding decor [title] Consider the season. [step] When you're choosing the colors to use in your color blocked wedding, consider what season you're getting married in. This will give you a good place to start for selecting colors.\n\nA. Typically, wedding colors are in season during the summer and fall months. In warmer months, you may opt for colors like dark blue, gray, or light blue.\nB. If you're booking and preparing for a wedding that's going on until spring, you might want to start with a sunset theme. If you're starting for a weekend or holiday wedding, however, it may be a better idea to get your colors in season.\nC. [substeps] Winter weddings tend to use whites, blues, purples, and sometimes a pop or two of red. Spring wedding color schemes usually use pastel colors-pinks, purples, yellows, oranges, blues, and greens.\nD. [substeps] For example, during a warm spring wedding, the wedding colors are usually red, pink, and green. Autumn color blocking for a warmer wedding is often a good option.","prompt":"[header] How to use color blocking in your wedding decor [title] Consider the season. [step] When you're choosing the colors to use in your color blocked wedding, consider what season you're getting married in. This will give you a good place to start for selecting colors.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5939,"error":null,"output":"C"}
{"question_id":"hellaswag:00006416","item_index":340,"attempt":0,"prompt_hash":"a03b31dcc0b5ea93ff34030237a2488ffd7668d9ed953a7b743587fa658d20e7","question":"[header] How to wear slouchy boots [title] Opt for short slouchy boots for everyday wear. [step] Mid-calf slouchy boots are great for everything from going to work to running errands. Pair them with jeans or leggings to keep your look casual.\n\nA. It's also a great idea to look to find shorter boots that don't slip off of your feet too easily. [substeps] A quality alternative is to opt for inchino boot style when you go snow-climbing.\nB. [substeps] You could even go for slouchy ankle boots if you want a shorter pair. [title] Select tall slouchy boots if you like to dress up.\nC. [substeps] Once you wear a longer boot, you can pair it with flowing leather or other lightweight material like denim and tucked-in jeans. For example, wear a short knee-length dress and leather leggings in a neutral color that stops at mid-calf.\nD. Although they give your legs a little more control during everyday wear, you also want to consider the weather when you wear them. [substeps] If you want to enhance your look, a longer leg boot may be too heavy for work or school.","prompt":"[header] How to wear slouchy boots [title] Opt for short slouchy boots for everyday wear. [step] Mid-calf slouchy boots are great for everything from going to work to running errands. Pair them with jeans or leggings to keep your look casual.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5937,"error":null,"output":"B"}
{"question_id":"hellaswag:00007175","item_index":341,"attempt":0,"prompt_hash":"71d80c25589fd1d5639a911dd717180e37eb7bbff2dbccf402964b98cc9aabde","question":"[header] How to dye your hair brown after it has been dyed black [title] Buy the right products. [step] There are two types of shampoos that will help take color off of your hair. Clarifying shampoo is rich in color-stripping ingredients and dandruff shampoo also helps strip color from your hair.\n\nA. [substeps] Clarifying shampoo also washes out dirt and oil from your hair and helps your hair dry faster. You can find clarifying shampoo at most pharmacies.\nB. [substeps] Paste shampoo is meant to promote shine, while goo gone shampoo is meant to take color off of your hair. [title] Rinse out excess oil and finish with a protective conditioner to boost shine.\nC. These shampoos will help break down the color on your hair and get your hair back to its original color. You can also buy conditioner that is not safe for color treated hair.\nD. You can find these shampoos at hairsalas or hair-washing labels. [substeps] Clarifying shampoo should be used sparingly and never use more often than recommended.","prompt":"[header] How to dye your hair brown after it has been dyed black [title] Buy the right products. [step] There are two types of shampoos that will help take color off of your hair. Clarifying shampoo is rich in color-stripping ingredients and dandruff shampoo also helps strip color from your hair.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5938,"error":null,"output":"C"}
{"question_id":"hellaswag:00002885","item_index":342,"attempt":0,"prompt_hash":"616deb174a8cc6f3a4e3a2a60c1630797fb7ec741f29cbd543b20d189dd292ff","question":"He looks in the mirror then reaches for the face wash and puts some in his hand. He throws the face wash on the washing machine. he\n\nA. walks into the office.\nB. throws the tube on the floor to make it sink.\nC. puts his head in the sink and washes his face angrily.\nD. turns the water on to clean the mouth and eyes.","prompt":"He looks in the mirror then reaches for the face wash and puts some in his hand. He throws the face wash on the washing machine. he","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6633,"error":null,"output":"C"}
{"question_id":"hellaswag:00000603","item_index":343,"attempt":0,"prompt_hash":"986b721083f0f92352df1eaba4f14bf99e700bf443b7a35f92a25abc2cdc66ee","question":"The girls clean their faces while laughing. One girl dries her face on a towel hanging on the wall. the second girl\n\nA. reaches for her chin and makes a face.\nB. puts makeup on her face.\nC. plays with her hair, then walks on the floor.\nD. dries her face on the towel.","prompt":"The girls clean their faces while laughing. One girl dries her face on a towel hanging on the wall. the second girl","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6633,"error":null,"output":"D"}
{"question_id":"hellaswag:00002954","item_index":344,"attempt":0,"prompt_hash":"aeb2bdea70c01dc3961eb18417be89db2432b04ebe52193431b652730ae03cb3","question":"Between matches the people all go out into the grass and put the patches of grass back into the holes that are missing them, and the polo game continues for another round. the polo match ends and prince harry and prince william\n\nA. and stuffed guests walk off the field, and the young men from the polo team talk to people who played the game, and laugh.\nB. return to their machine.\nC. are seen separately receiving a white box from a man, while they talk and shake hands as people clap and take pictures.\nD. hit the ball back and forth with their fans, and the players have a great time celebrating.","prompt":"Between matches the people all go out into the grass and put the patches of grass back into the holes that are missing them, and the polo game continues for another round. the polo match ends and prince harry and prince william","gold":"C","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6632,"error":null,"output":"D"}
{"question_id":"hellaswag:00007526","item_index":345,"attempt":0,"prompt_hash":"f3e1c82217ba06f0c0dacb5a96356c0b67e8beab659262cf42a8be553519ef66","question":"[header] How to choose a red dress [title] Determine your skin tone. [step] Your skin tone has a big impact on what colors looks best on you, so it's important to know what type you have before choosing a shade of red. Skin tones tend to fall into three categories: warm, cool, and neutral.\n\nA. Warm skin brings more colour into your color spectrum than cool color brings. Cool skin often has less hue and brighter colors, but cool skin tends to be prettier and have deeper shades that add color more quickly.\nB. Your skin tends to fall into whichever category works for you most. [substeps] Warm skin tones are usually cool colors.\nC. Warm skin tones tend to look best and cover more skin in one color. Cool skin tones are warmer colors that cover more skin, but are still more prone to darkening.\nD. [substeps] People with warm skin tones have yellow, peachy or golden undertones and tend to tan more easily. The veins on their inner arms appear green.","prompt":"[header] How to choose a red dress [title] Determine your skin tone. [step] Your skin tone has a big impact on what colors looks best on you, so it's important to know what type you have before choosing a shade of red. Skin tones tend to fall into three categories: warm, cool, and neutral.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6633,"error":null,"output":"D"}
{"question_id":"hellaswag:00001648","item_index":346,"attempt":0,"prompt_hash":"2d345e7e4eca0249ec8e8c1669e059cd7ceb0ae0efabb0f7d96f5e99c4011cf7","question":"Young man is in the middle of stage and its spreading white cram on his hands and walks making deep beraths. man\n\nA. is sitting in the middle of stage and pour green mascara on his hands leaving it purple and wash it out with soap and water.\nB. is writing people a paper saying white cram.\nC. is ready to lifting weight in middle of a white stage and people behind jim is clapping.\nD. is saying ruh and using the dictaphone attached to the small black screen.","prompt":"Young man is in the middle of stage and its spreading white cram on his hands and walks making deep beraths. man","gold":"C","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6632,"error":null,"output":"A"}
{"question_id":"hellaswag:00002306","item_index":347,"attempt":0,"prompt_hash":"e4517f50ceb56111db357929458c5c0370f87faa68802fbf46a8ac7ac7f0d56d","question":"A woman is handed an ice cream cone. Two police officers order ice cream and wait for it. a customer\n\nA. removes a vanilla cone from the ice cream dish.\nB. is seen on top of the cone.\nC. exchanges ice cream cones with a woman.\nD. appears irritated by the officers.","prompt":"A woman is handed an ice cream cone. Two police officers order ice cream and wait for it. a customer","gold":"D","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6757,"error":null,"output":"C"}
{"question_id":"hellaswag:00008154","item_index":348,"attempt":0,"prompt_hash":"adaca44936f23f66fbb3202c391c9c4368b46939f90dab2b9d999ba133492f1a","question":"[header] How to make homemade cat repellent [title] Add the essential oil to a spray bottle. [step] For the repellent, you'll need a 2-ounce (59 ml) glass spray bottle. Squeeze 2 drops of lemon essential oil , 2 drops of wild orange essential oil, and 2 drops of lavender essential oil to the bottle.\n\nA. Shake the spray bottle until all of the ingredients are well blended. [substeps] If you don't have enough supplies, you can use cuticles, gum, or some other item as a repellent.\nB. Gently shake the bottle to combine all the ingredients. [title] Spray the repellent on your cat's skin.\nC. [substeps] Cats are more sensitive to scents than humans, so essential oils with strong odors, such as citrus and lavender, can help repel them. You can substitute lime, peppermint, and/or eucalyptus for the lemon, wild orange, and lavender if you prefer.\nD. Use a sprayer to spray the essential oil on a small piece of a cotton ball. Leave the spray on for 1-2 hours, or until the oil is thoroughly infused.","prompt":"[header] How to make homemade cat repellent [title] Add the essential oil to a spray bottle. [step] For the repellent, you'll need a 2-ounce (59 ml) glass spray bottle. Squeeze 2 drops of lemon essential oil , 2 drops of wild orange essential oil, and 2 drops of lavender essential oil to the bottle.","gold":"C","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6690,"error":null,"output":"B"}
{"question_id":"hellaswag:00009441","item_index":349,"attempt":0,"prompt_hash":"9c3a14acf1d336b88dc252ff504b5fb6d4277ded5b2f55a4a45fd5ca1903f9ae","question":"[header] How to decide which piercing is best for you [title] Consider the visibility of the piercing. [step] Ear and facial piercings are very visible, which can cause problems at school or work for some people. If you're thinking about getting a piercing, make sure you won't have to take it out at school or at your job.\n\nA. [substeps] If you're set on getting one, give some thought to covering your piercing in the meantime. Sometimes, schools will allow piercings to be covered with a small bandage while it heals.\nB. [substeps] Conceal your piercing with a square ring or other piece that most people normally won't notice. Keeping your piercing hidden may help keep a wide variety of people from noticing it.\nC. [substeps] In-person piercings are more effective as they will more comfortably cut through the skin. However, ear piercings can be uncomfortable for you, unless you're confident that they won't bruise your ears.\nD. [substeps] Ear and facial piercings are very visible. Since they're so visible, think about what sort of images you want your piercing to capture.","prompt":"[header] How to decide which piercing is best for you [title] Consider the visibility of the piercing. [step] Ear and facial piercings are very visible, which can cause problems at school or work for some people. If you're thinking about getting a piercing, make sure you won't have to take it out at school or at your job.","gold":"A","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":6689,"error":null,"output":"D"}
{"question_id":"hellaswag:00005905","item_index":350,"attempt":0,"prompt_hash":"0dfcec2798945eb8c7dd96f38674ea5573475b37a7c94438fbb87605cbecf04d","question":"[header] How to help children with adhd sleep better [title] Set a firm bedtime. [step] A good way to help your child with adhd sleep better is to get him in the habit of going to bed at the same time. Pick a realistic bed time for your child's schedule and your family activities.\n\nA. [substeps] Maybe he goes to bed about 3 or 4 hours before his bedtime. If you find that he spends a lot of time outside throughout the day, you may try to adjust your overnight schedule to accommodate such a change.\nB. [substeps] Suggest doing household chores that are meant for him and that you can see him perform. For example, you might tell your child that bedtime is for you and he can do the dishes and the other chores.\nC. Remember this also may help him to sleep easier at night, and more comfortably on weekends. [substeps] It's ok if the child wakes you up by 11 am and sleeps into the wee hours on weekends.\nD. Make this your child's bedtime is the same time every night, even on the weekend. [substeps] To help your child stick to this bedtime, remind your child it's time for bed starting at an hour before.","prompt":"[header] How to help children with adhd sleep better [title] Set a firm bedtime. [step] A good way to help your child with adhd sleep better is to get him in the habit of going to bed at the same time. Pick a realistic bed time for your child's schedule and your family activities.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6689,"error":null,"output":"D"}
{"question_id":"hellaswag:00009485","item_index":351,"attempt":0,"prompt_hash":"a1b0b218a51a10be616c759b5b5d61567c66a8f1154baacb1eaa37fc44c4bf95","question":"[header] How to schedule your college classes effectively [title] Look at the course schedule often and early. [step] Familiarize yourself with the courses your college is offering each semester. The sooner you know what courses are being offered, the sooner you can register for those classes.\n\nA. [substeps] It will be a hassle to enroll in a class that is already full. You may have to petition the professor or the department or, unfortunately, wait until a different semester or year to take the course.\nB. [substeps] Choose classes that interest you, and how challenging it can be to consider and participate in them. [title] Pay attention to schedule and class consistency.\nC. This will also make visiting your old college easier. [title] Choose your major, classes and assignments carefully.\nD. Review course schedules and times every year to see if there are exceptions. [substeps] Checking out class schedules can help you plan your classes closer to your deadlines.","prompt":"[header] How to schedule your college classes effectively [title] Look at the course schedule often and early. [step] Familiarize yourself with the courses your college is offering each semester. The sooner you know what courses are being offered, the sooner you can register for those classes.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":6689,"error":null,"output":"A"}
{"question_id":"hellaswag:00006571","item_index":352,"attempt":0,"prompt_hash":"8529a69869648e28656a89c009492676958715e73e98b5d4a37baddaa75468de","question":"[header] How to split expenses as a couple [title] Determine what expenses qualify as shared. [step] Sometimes it is easy to identify a shared expense. For instance, heating, water, and electricity are house-wide expenses and you will both, presumably, enjoy their use in approximately equal amounts.\n\nA. However, it may not be possible to work together with such conditions, and it may be unnecessary. For example, the value of your home will be raised to the $20 that wasn't available to make cable and television more sustainable.\nB. But it might be harder to justify sharing other expenses. If you have tv service at your home, for instance, but only one of you watches tv, it makes little sense to split that particular expense.\nC. Other times, as a couple, even though one pays, the other only receives reimbursement. [substeps] One can agree on a separate expense if one is in touch with the other.\nD. Keep in mind that some of these expenses may not be within the home. As a family, having these costs put together is a good indicator of the financial possibility you have of splitting taxes.","prompt":"[header] How to split expenses as a couple [title] Determine what expenses qualify as shared. [step] Sometimes it is easy to identify a shared expense. For instance, heating, water, and electricity are house-wide expenses and you will both, presumably, enjoy their use in approximately equal amounts.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5069,"error":null,"output":"B"}
{"question_id":"hellaswag:00008385","item_index":353,"attempt":0,"prompt_hash":"40d678a7678dc2d73909cb040b55701dfa68ec4d8eb7171e23e77d3d6d588654","question":"[header] How to make organic sunscreen [title] Fill the bottom portion of a double boiler with a few inches (10 to 20 centimeters) of water. [step] Place the double boiler on the stove and heat over medium-low heat, enough to make the water hot but not boiling. Do not attempt to mix your organic sunscreen over direct heat using a standard sauce pot.\n\nA. [title] Place the sunscreen into the double boiler and allow it to heat. [step] When the water boils, turn the burner up to medium-high.\nB. The pot of water should turn into a big ball and create a steaming motion as you heat the sunscreen. [substeps] When you buy a bottle of organic sunscreen, look for small bottles that you can fit all your ingredients and oil to.\nC. [title] Place the beeswax and coconut oil into the upper portion of the double boiler. [step] Slowly heat and stir the two ingredients together with a heat resistant spoon until both have melted into a liquid state and have combined.\nD. [substeps] The maximum amount of water can be added can be substituted for boiling water. For some instructions, please check the guidelines below for specific amounts, as well as how much water you should use.","prompt":"[header] How to make organic sunscreen [title] Fill the bottom portion of a double boiler with a few inches (10 to 20 centimeters) of water. [step] Place the double boiler on the stove and heat over medium-low heat, enough to make the water hot but not boiling. Do not attempt to mix your organic sunscreen over direct heat using a standard sauce pot.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5068,"error":null,"output":"C"}
{"question_id":"hellaswag:00006887","item_index":354,"attempt":0,"prompt_hash":"540aed9f636264fcc731050c7a6f6f801cfa1d42fb3f70e0b6436621f8363fa2","question":"[header] How to do external car detailing properly [title] The first thing you need to do properly is to wash down your car effectively. [step] There is no point trying to do external car detailing if you are just going to end up scratching your duco even more because you left dirt on your car. [substeps] You must first hose down your car with firm pressure.\n\nA. This will remove most of the dirt from your engine and keep the engine running smoothly. Use high pressure washing machines to wash down your car.\nB. Then you must begin to siphon off dirt. Once you have removed as much dirt as possible, you can then return to the vehicle to pick up trash.\nC. Run a hose from your carburetor to the top of the engine block, wait for five minutes, then turn off the water and let the air out of the radiator. Remove the old gaskets and begin water washing down your engine cover.\nD. Don't use a laser type wash from your hose, but rather a small funnel. Always start from top and work your way down.","prompt":"[header] How to do external car detailing properly [title] The first thing you need to do properly is to wash down your car effectively. [step] There is no point trying to do external car detailing if you are just going to end up scratching your duco even more because you left dirt on your car. [substeps] You must first hose down your car with firm pressure.","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5068,"error":null,"output":"A"}
{"question_id":"hellaswag:00005149","item_index":355,"attempt":0,"prompt_hash":"dede61cfc75998f44ff50bc8c33f03497710aa2be1adfc7da8a564b149070c97","question":"[header] How to cite legal research [title] Write the name of the case. [step] A legal citation to a court case begins with the title of the case itself, as it appears in the case reporter you're using. Generally, you should only use last names or business names.\n\nA. Don't use the first word of the case you want to write with your first name. [substeps] If you don't know the full legal name of the case you're citing, skip ahead to the footnote and save it.\nB. However, you can add business names if you want or wish. [substeps] For example, using \" smith vs.\nC. [substeps] For example, you could write, \" this criminal defense \" rather than \" plaintiff (or defendant), \" in this case. [title] Collect citations from adaddendum.\nD. [substeps] Don't include the phrase \" et al. \" to indicate there is more than one party to the suit.","prompt":"[header] How to cite legal research [title] Write the name of the case. [step] A legal citation to a court case begins with the title of the case itself, as it appears in the case reporter you're using. Generally, you should only use last names or business names.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5070,"error":null,"output":"D"}
{"question_id":"hellaswag:00000311","item_index":356,"attempt":0,"prompt_hash":"60d6ab21b96ce48ebda95ab423384bfadc7cc7eb1234a40c493f1385023fd7a5","question":"A large mat is seen with a bar on top and two people watching on the sides. A man is then seen running in from the side. the man\n\nA. takes his jacket off and begins beating a bag over and over.\nB. jumps over the beam and stands up on the mat.\nC. then throws a ball down onto the mat.\nD. then performs several more lifts on the bar.","prompt":"A large mat is seen with a bar on top and two people watching on the sides. A man is then seen running in from the side. the man","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5065,"error":null,"output":"B"}
{"question_id":"hellaswag:00000157","item_index":357,"attempt":0,"prompt_hash":"48b969707c075c1f93398c6946721e6009c1cb05a3c7d530395d4bac4f18510a","question":"A young man plays the violin on a sidewalk for people passing by in downtown area. two students\n\nA. of swimming are in the swimming pool watching him play the violin.\nB. play the violin on front people who pass by in street, looking at the violin.\nC. are playing drums in a park while other people walk past on sidewalk.\nD. walk by behind the musician while he plays.","prompt":"A young man plays the violin on a sidewalk for people passing by in downtown area. two students","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5065,"error":null,"output":"D"}
{"question_id":"hellaswag:00008516","item_index":358,"attempt":0,"prompt_hash":"c592e0a9e7e77c2b9df30d7810bfcab01b27055630419321d88f23de3d069ff1","question":"[header] How to buy shoes [title] Buy shoes from a brand known for quality and comfort. [step] Many well-established footwear brands are known for their reliable, functional footwear. Buy a pair of shoes from one of these brands to make sure that you are getting a high quality product.\n\nA. Most of these companies have built their reputations on years of good customer service as well, so these shoes are likely to be covered by a generous warranty. [substeps] Find quality footwear brands by reading online reviews.\nB. [title] Talk to your shoe shop employees about new shoes. [step] This could change their shoes line or lower their sales pitch.\nC. Check the store's label for details and tips on sneakers or other shoes. [substeps] Children should not wear their sneakers too frequently and normal public footwear should not be worn as shoes; however, kids who wear their sneakers for a variety of periods should generally wear sturdy shoes.\nD. Once you purchase shoes you would be prepared to spend most of your time living with them, although that doesn't mean that they will last forever. [substeps] If you're having a hard time finding good sneakers, try looking for good pairs of shoes at the store.","prompt":"[header] How to buy shoes [title] Buy shoes from a brand known for quality and comfort. [step] Many well-established footwear brands are known for their reliable, functional footwear. Buy a pair of shoes from one of these brands to make sure that you are getting a high quality product.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5061,"error":null,"output":"A"}
{"question_id":"hellaswag:00003937","item_index":359,"attempt":0,"prompt_hash":"969f3949b7581f8a04e51a54c7be2b43f22a4f6bf537df862536b26012dab92f","question":"[header] How to make a bridal shower fun [title] Speak to the bride before you begin planning. [step] Traditionally, the bride is not involved with the shower, but if she is hard to please, it will be easier to talk through any worries or ideas she might have at the beginning of the process and avoid confrontations that could ruin the day. [title] Decide if a couple's shower would be more fun.\n\nA. [step] If the couple's wedding is impromptu, a shower would be much better. However, perhaps a more intimate shower might be even more fun.\nB. [step] You can co-plan with the best man and reduce the formality of the event, leading to a more enjoyable time. [title] Determine the budget for the shower to reduce any financial worries.\nC. [step] At a wedding, some glitz and glamour might also be involved. Decide if a couple's shower could be more fun if you are deciding who's coming to collect the bouquet.\nD. [step] If the bride has an emphasis on her health, then one of these shower items may be appropriate. If so, then go on ahead with the shower.","prompt":"[header] How to make a bridal shower fun [title] Speak to the bride before you begin planning. [step] Traditionally, the bride is not involved with the shower, but if she is hard to please, it will be easier to talk through any worries or ideas she might have at the beginning of the process and avoid confrontations that could ruin the day. [title] Decide if a couple's shower would be more fun.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5059,"error":null,"output":"B"}
{"question_id":"hellaswag:00004778","item_index":360,"attempt":0,"prompt_hash":"99169be8747588d327a22c8895c4614143d0b83744dc32f7b2da66d01b147acd","question":"[header] How to prepare the breast for breast feeding [title] Massage, but do not \" rough up, \" your breasts. [step] Massaging your breasts will help you relax and prepare you in case you need to manually express milk for the baby. [substeps] The massage should be gentle and not painful.\n\nA. [title] Get up and move around as you usually would. [step] It may seem awkward to you, but to a baby it may be easier to learn how to do by simply sitting or squatting.\nB. You should massage your breasts from either side to generate the stimulation you will experience by feeding. Rubbing your breasts with your fingertips will generate the lubricant so you can rub it into your breasts quickly.\nC. Start above the breast and stroke with a circular motion while moving towards the nipple. Then move again to the outside of the breast in a different area and repeat, moving towards the nipple.\nD. After massaging, grasp the breast with your fingertips and gently pat your breast dry with a towel. Use the hand that was not licking your breasts or massaging gently, or tap gently with two fingers.","prompt":"[header] How to prepare the breast for breast feeding [title] Massage, but do not \" rough up, \" your breasts. [step] Massaging your breasts will help you relax and prepare you in case you need to manually express milk for the baby. [substeps] The massage should be gentle and not painful.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5059,"error":null,"output":"C"}
{"question_id":"hellaswag:00000504","item_index":361,"attempt":0,"prompt_hash":"4efb1eeb26412c5ce11fa7a682ffd082f4e0d2c240f3bb2a84a16deae51905a1","question":"A is standing in front of a balance bar. the girl\n\nA. begins to walk up it and then turns back around.\nB. jumps onto the bar.\nC. begins to do several hand tricks as the camera focuses.\nD. talks to the camera many times.","prompt":"A is standing in front of a balance bar. the girl","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5060,"error":null,"output":"B"}
{"question_id":"hellaswag:00009068","item_index":362,"attempt":0,"prompt_hash":"f2a4d9b9dc0c878c625de33f8da3b1ace1245f643419700c8e4542938a840d21","question":"[header] How to rotate screen on iphone [title] Tap your iphone's home button. [step] Usually, you can rotate the screen on your iphone simply by disabling the default rotation lock, then turning the iphone on its side. [substeps] You can also tap your iphone's lock button, since your goal here is to \" wake up \" your iphone's screen.\n\nA. [title] Swipe up from the bottom of your screen. [step] This will open the control center, from which you can enable or disable rotation lock.\nB. [title] Swipe down from the bottom of the screen. [step] This will bring you to the home screen.\nC. It doesn't need to be a screen lock; it can also be a home screen switch. [title] Tap a screenshot of your camera lens.\nD. [title] Swipe your finger up and down on the screen, leading the screen to rotate in a clockwise direction. [step] Hold the home button down while flicking the screen.","prompt":"[header] How to rotate screen on iphone [title] Tap your iphone's home button. [step] Usually, you can rotate the screen on your iphone simply by disabling the default rotation lock, then turning the iphone on its side. [substeps] You can also tap your iphone's lock button, since your goal here is to \" wake up \" your iphone's screen.","gold":"A","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5058,"error":null,"output":"A"}
{"question_id":"hellaswag:00007262","item_index":363,"attempt":0,"prompt_hash":"0aec3fc18b900bb45755d17fa632b82d51cbc84494f73bd5189ceb2813bc9f79","question":"[header] How to care for a pug [title] Give their facial folds a weekly clean. [step] Your pug's facial folds can hold onto food, saliva, dirt, and other debris that may be smelly and irritating. Use a cotton swab dipped in warm water to clean them out.\n\nA. With no cleaning supplies, wash them once a day with warm water and soapy water in between. [substeps] If you notice any pink or red spots or discharge on your pug's skin after cleaning, it might be a sign of a bacterial infection.\nB. Use warm water only if they are very dirty-which they will eat, but that is not always necessary). Let them dry with a towel, and use items like paper towels to dry their skin if it is really dirty.\nC. Use a soft, nonabrasive cloth to gently scrub out any food or plaque from their faces. [substeps] Washcloth cleanses are also great for wash-ants, for instance.\nD. Run the swab along the folds around your dog's nose and eyes, being careful to not let the swab make contact with their eyes, nostrils, or mouth. [substeps] Don't leave the folds damp, as this could cause further irritation.","prompt":"[header] How to care for a pug [title] Give their facial folds a weekly clean. [step] Your pug's facial folds can hold onto food, saliva, dirt, and other debris that may be smelly and irritating. Use a cotton swab dipped in warm water to clean them out.","gold":"D","extracted_answer":"D","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5060,"error":null,"output":"D"}
{"question_id":"hellaswag:00003392","item_index":364,"attempt":0,"prompt_hash":"3950802740679f5b3eb51bb829038fef54bd4f6757c57677d938285938e34d85","question":"[header] How to know if your dog likes you the best [title] Notice that your dog will follow you around a lot. [step] One way to prove that a dog likes you most is when they're around you a lot. So keep an eye out to see if your dog likes to be around you.\n\nA. Notice that your dog follows you around the house everyday when you're out and about. While you may not feel like it, sometimes spending a lot of time around a dog can make one feel comfortable around you.\nB. [title] Keep an eye out for any physical behaviors they may have. [step] A good example of this behavior is if they press their butt up against your thigh and check to see what you're wearing.\nC. [title] Find out if your dog likes to play with you. [step] If it is a dog who loves toys, play with them and if they're really excited to play, then they enjoy you playing with them.\nD. [title] Look for cues that your dog may be flirty. [step] One good signal to recognize when a dog likes you is if your dog gives you a significant amount of petting or stares at you in intimate moments.","prompt":"[header] How to know if your dog likes you the best [title] Notice that your dog will follow you around a lot. [step] One way to prove that a dog likes you most is when they're around you a lot. So keep an eye out to see if your dog likes to be around you.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5057,"error":null,"output":"C"}
{"question_id":"hellaswag:00007356","item_index":365,"attempt":0,"prompt_hash":"7b539fa27c8fcef5f48e566c1956bc8e02d2346b9cd3c97c05a6a1f7cd36fbae","question":"[header] How to make goat cheese [title] Place your milk over medium heat. [step] Measure your milk into a saucepan. Place it on the stove over medium heat.\n\nA. Make sure it is at a medium heat. Heat the milk at medium until it reaches a heat temperature.\nB. [title] Heat the milk until 180 degrees fahrenheit (87 degrees celsius). [step] Stir the milk frequently to make sure it heats evenly throughout.\nC. Let it heat until it reaches the temperature at which you want it. This should take approximately 5 to 7 minutes.\nD. Stir it every two minutes, or until it reaches a boil. When milk boils or bubbles, it needs to be warmed so that it is completely dissolved.","prompt":"[header] How to make goat cheese [title] Place your milk over medium heat. [step] Measure your milk into a saucepan. Place it on the stove over medium heat.","gold":"B","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5060,"error":null,"output":"C"}
{"question_id":"hellaswag:00000537","item_index":366,"attempt":0,"prompt_hash":"844e706987cebcf01ecd3d0faa481832f58209529d33d34c8285704cfae03d3e","question":"A couple of men are standing on an iced over lake. they\n\nA. are engaged in a game of volleyball.\nB. jump off of a bridge onto a large rock.\nC. put the tubing into the water, led by a boy.\nD. have made holes through the ice.","prompt":"A couple of men are standing on an iced over lake. they","gold":"D","extracted_answer":"A","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5058,"error":null,"output":"A"}
{"question_id":"hellaswag:00006286","item_index":367,"attempt":0,"prompt_hash":"8ee9b7bad0ee4af68088b1d6e53d3a7850c62f5214818b35b4a955be29e19b98","question":"[header] How to hide keloid scars with makeup [title] Choose a green-undertone concealer to neutralize redness. [step] If your scars are red or pink, selecting a concealer color on the opposite end of the color wheel will make them look less inflamed. The concealer may look very green in the packaging, but it will turn into a flesh tone when applied.\n\nA. [substeps] Go for shades of green that complement the skin of your face. Green concealer works for both dark and pale skin, but any purple can work.\nB. Most concealers of this type will advertise that they are \" redness reducing. \" [substeps] Similarly, if your keloid scars are more yellowish, look for a concealer with violet undertones.\nC. Go for a natural shade of green that closely matches your skin so you can reduce redness. [substeps] If you decide to use green makeup, make sure you choose a color that will produce good results.\nD. [substeps] Try foundation with a violet undertone under the center of your face to create the illusion of shading. Let the foundation dry before applying your concealer to cover dark circles or pimples.","prompt":"[header] How to hide keloid scars with makeup [title] Choose a green-undertone concealer to neutralize redness. [step] If your scars are red or pink, selecting a concealer color on the opposite end of the color wheel will make them look less inflamed. The concealer may look very green in the packaging, but it will turn into a flesh tone when applied.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5057,"error":null,"output":"B"}
{"question_id":"hellaswag:00009149","item_index":368,"attempt":0,"prompt_hash":"a4cce50769b178be04d87244b940c7941ffda03875856d8e3a725a1f34a81e99","question":"[header] How to treat a turf burn [title] Clean the burn immediately. [step] Because turf burns tend to get dirt and other debris in them, it's important to clean them right away. Wash the burn with a mild soap and warm water, or use a.\n\nA. You should also soak the burn in cool water in the shower, or inside. [substeps] Make sure to wear gloves as you wash the area.\nB. This will help kill any foreign matter and clean the burn as well. [substeps] Rinse the burn off as soon as it starts to wash away.\nC. Rinse the burn with plenty of warm water to wash out any debris, and gently pat it with clean, moist, lint-free gauze. For larger pieces of debris, use clean tweezers to remove them.\nD. Do not use alcohol or any other nonflammable substances. [title] Keep soft tissues in the burn area.","prompt":"[header] How to treat a turf burn [title] Clean the burn immediately. [step] Because turf burns tend to get dirt and other debris in them, it's important to clean them right away. Wash the burn with a mild soap and warm water, or use a.","gold":"C","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5057,"error":null,"output":"C"}
{"question_id":"hellaswag:00000014","item_index":369,"attempt":0,"prompt_hash":"086f6fc8c22211a35b807d4df0a572b310e04640b26363d99afe00b50898864c","question":"A person is playing bagpipes out in a park. A man and two boys watching from a far. the camera\n\nA. shoots areas all around the park while the bagpipes play.\nB. is covered to the boy that is playing there.\nC. then goes black and we see a black opening scene.\nD. watching the boy and boy as they play a game of drums.","prompt":"A person is playing bagpipes out in a park. A man and two boys watching from a far. the camera","gold":"A","extracted_answer":"C","choices":null,"choice_scores":null,"score_normalization":null,"score":0,"passed":false,"latency_ms":5058,"error":null,"output":"C"}
{"question_id":"hellaswag:00007962","item_index":370,"attempt":0,"prompt_hash":"c08b1f27bb758f432b8764a45e0cace92d43b88db3d20816570860d08300e4e8","question":"[header] How to braise celery [title] Wash and chop the celery. [step] Start with eight stalks of celery. Thoroughly scrub the celery, running it under water to clean it.\n\nA. The extra water will help ensure that the celery isn't so fatty that it burns while you're braising it. Place the celery in the bowl you'll be braising it in, and rub it vigorously with your hands or a vegetable peeler to remove dirt and grit.\nB. Cut off the brown ends, and peel off the extra chewy part on the outside of the celery if you prefer. Chop the celery into 1-inch pieces.\nC. Trim the celery just before you add it to a salad spinner. Then, chop each celery stalk into bite-sized pieces.\nD. Chop the celery into pieces or chunks. If your kitchen is already equipped with good braising equipment, you can simply prepare the celery in batches to keep it from getting too soggy.","prompt":"[header] How to braise celery [title] Wash and chop the celery. [step] Start with eight stalks of celery. Thoroughly scrub the celery, running it under water to clean it.","gold":"B","extracted_answer":"B","choices":null,"choice_scores":null,"score_normalization":null,"score":1,"passed":true,"latency_ms":5059,"error":null,"output":"B"}