Across sub-Saharan Africa, climate change is leading to increasingly erratic weather patterns that challenge farming practices, particularly for smallholder farmers. Preparing farmers for these changes and increasing their resilience to extreme weather events is critical for food security in areas where populations are increasing. The El Niño event of 2015/16 led to drought conditions in Malawi which are expected to become more normal in the future. This resulted in widespread crop failure and the need for external food aid. The experiences of Malawian farmers during this time creates an opportunity to identify areas where adaptations in land management practices as part of resilience building initiatives can prepare farmers for future climates. This paper presents results of household surveys and interviews of 201 farmers from a case study in southern Malawi. Half of the farmers surveyed practice Conservation Agriculture (CA), a Climate Smart Agriculture technology that increased resilience to this drought event. The majority of households relied on agriculture for all their livelihood streams, indicating that diversification away from sole dependence on agriculture would increase resilience. Our study shows that poorer, female farmers are less likely to practice CA than wealthier male farmers. Results also illustrate that while farmers had access to seasonal weather forecasts, a key tool to guide land preparation and planting, they remained reluctant to believe them or to amend cropping or land management practices. Agricultural extension services within Malawi can play a vital role in preparing farmers for future extreme weather events and ensuring forecast communication link to predicted agricultural impacts and land management actions for building resilience into agricultural systems. Extension services need to focus on supporting poorer female farmers to adopt CA practices and providing farmers with the tools and knowledge to respond effectively to seasonal and sub-seasonal climate information.