There are several types of housing possibilities for graduate students at Penn.
There is on-campus housing in the University's graduate towers at Sansom Place, located on Chestnut Street between 36th and 37th Streets (four blocks from DRL, which is where the Penn Mathematics Department is located). More information about Sansom Place is available on the web, by email at living@pobox.upenn.edu, or by phone (215-898-8271).
Just off campus, between 37th and 38th Streets on Chestnut Street (five blocks from DRL) is Philadelphia's International House. This is one of a network of International Houses throughout the United States. It is run independently of Penn, and houses students from Penn, Drexel University, and other nearby academic institutions, mostly from outside the US. Many cultural events take place there, most of them open to the community. Information is available on the web or by phone (215-895-6535).
Off campus, the two most popular locations for Penn's math grad students are University City and Center City.
University City is the neighborhood of West Philadelphia adjacent to Penn's campus. Math grad students who live off campus in University City tend to live roughly in an area bounded on the east by the Penn campus, on the west by 46th Street, on the north by Locust Street, and on the south by Baltimore Avenue. This is a residential area with tree-lined streets and old stone and brick houses, and it is a pleasant walk from there through campus to the Math Department. Nearby there are supermarkets, movie theaters, restaurants, and book stores.
Center City is the downtown area of Philadelphia, east of the Schuylkill River. It is easy to reach from Penn on foot (though one can bike or take a bus); it's about a ten minute walk east from DRL to the western edge of Center City. Most of the math grad students who live in Center City live in an area bounded on the west by the Schuylkill River, on the east by 15th Street, on the north by Chestnut Street, and on the south by Bainbridge Street. Center City is a denser area than University City, and is also somewhat more expensive. It has both low and high rise apartments, and both residential and commercial streets. It has many restaurants and cafes, book stores, hotels, museums, venues for many types of music, etc.
Some grad students live off-campus in various other locations: in Powelton Village (an area about six blocks north of Penn), in more distant locations within Philadelphia (e.g. Northeast or Northwest Philadelphia), or in the suburbs. There is a fairly extensive system of commuter trains to reach Penn from outlying areas, as well as a subway system and many buses in the city.
Information about off-campus living is available on Penn's web site. Penn's office of off-campus living is located at 4046 Walnut Street, just west of campus. The telephone number is 215-898-8500. Additional off-campus housing listings for University City are available from Campus Apartments; their phone number is 215-382-1300.
Some grad students have arranged for their housing before they arrive at Penn, via email and the web. Others arrive a bit early, and live briefly in temporary accomodations while looking for a more permanent place to live. Also, while some students live by themselves, others share apartments, often with other grad students. This tends to reduce the cost, and many students also prefer the experience.
More information about housing, the area surrounding the campus, etc. can be found on the campus page of Penn's web site.