Ideas are Nothing – Execution is Everything Optimize Data Transfers Over High Latency Networks
Oct 18

You should always avoid downloading data over a wireless network whenever possible. If there is a file on a server that your app needs and it’s infrequently updated, then a good approach would be to cache that file in your app and only download it when an update is detected. (You could also package a version of the file inside the app bundle to provide a quick startup time the first time the app is launched.)

There are several ways to solve this problem. The code below does a HTTP HEAD request, checks the returned "Last-Modified" header, and compares it with updated timestamp of the local file. (A slightly more efficient approach would be to send a "If-Modified-Since" header with your request. But there seems to be some issues with that, so proceed carefully with that approach.)

High-level steps:

  1. Create a HTTP HEAD request.
  2. Read the "Last-Modified" header and convert the string to a NSDate.
  3. Read the last modification timestamp of the local file.
  4. Compare the two timstamps.
  5. Download the file from the server if it has been updated.
  6. Save the downloaded file.
  7. Set the last modification timestamp of the file to match the "Last-Modified" header on the server.

Please note that the code uses synchronous requests to make the code shorter and better illustrate the point. Obviously you should not call this method from the main thread. Depending on your requirements, you could rewrite the code to use asynchronous requests instead.

- (void)downloadFileIfUpdated {
	NSString *urlString = ... your URL ...
	DLog(@"Downloading HTTP header from: %@", urlString);
	NSURL *url = [NSURL URLWithString:urlString];

	NSString *cachedPath = ... path to your cached file ...
	NSFileManager *fileManager = [NSFileManager defaultManager];

	BOOL downloadFromServer = NO;
	NSString *lastModifiedString = nil;
	NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:url];
	[request setHTTPMethod:@"HEAD"];
	NSHTTPURLResponse *response;
	[NSURLConnection sendSynchronousRequest:request returningResponse:&response error: NULL];
	if ([response respondsToSelector:@selector(allHeaderFields)]) {
		lastModifiedString = [[response allHeaderFields] objectForKey:@"Last-Modified"];
	}

	NSDate *lastModifiedServer = nil;
	@try {
		NSDateFormatter *df = [[NSDateFormatter alloc] init];
		df.dateFormat = @"EEE',' dd MMM yyyy HH':'mm':'ss 'GMT'";
		df.locale = [[[NSLocale alloc] initWithLocaleIdentifier:@"en_US"] autorelease];
		df.timeZone = [NSTimeZone timeZoneWithAbbreviation:@"GMT"];
		lastModifiedServer = [df dateFromString:lastModifiedString];
		[df release];
	}
	@catch (NSException * e) {
		NSLog(@"Error parsing last modified date: %@ - %@", lastModifiedString, [e description]);
	}
	DLog(@"lastModifiedServer: %@", lastModifiedServer);

	NSDate *lastModifiedLocal = nil;
	if ([fileManager fileExistsAtPath:cachedPath]) {
		NSError *error = nil;
		NSDictionary *fileAttributes = [fileManager attributesOfItemAtPath:cachedPath error:&error];
		if (error) {
			NSLog(@"Error reading file attributes for: %@ - %@", cachedPath, [error localizedDescription]);
		}
		lastModifiedLocal = [fileAttributes fileModificationDate];
		DLog(@"lastModifiedLocal : %@", lastModifiedLocal);
	}

	// Download file from server if we don't have a local file
	if (!lastModifiedLocal) {
		downloadFromServer = YES;
	}
	// Download file from server if the server modified timestamp is later than the local modified timestamp
	if ([lastModifiedLocal laterDate:lastModifiedServer] == lastModifiedServer) {
		downloadFromServer = YES;
	}

	if (downloadFromServer) {
		DLog(@"Downloading new file from server");
		NSData *data = [NSData dataWithContentsOfURL:url];
		if (data) {
			// Save the data
			if ([data writeToFile:cachedPath atomically:YES]) {
				DLog(@"Downloaded file saved to: %@", cachedPath);
			}

			// Set the file modification date to the timestamp from the server
			if (lastModifiedServer) {
				NSDictionary *fileAttributes = [NSDictionary dictionaryWithObject:lastModifiedServer forKey:NSFileModificationDate];
				NSError *error = nil;
				if ([fileManager setAttributes:fileAttributes ofItemAtPath:cachedPath error:&error]) {
					DLog(@"File modification date updated");
				}
				if (error) {
					NSLog(@"Error setting file attributes for: %@ - %@", cachedPath, [error localizedDescription]);
				}
			}
		}
	}
}

Line item notes:

03. See this post for the source to the DLog macro.

14. It’s always good practice to check if the returned object responds to a selector before you send a message to it. See this post for more details.

22. The "Last-Modified" timestamp on the server includes the name of the month as text. Set the locale of the date formatter to English (assuming that your server responds in English) otherwise the code will fail when run on devices in non-English countries. See this post for more details.

Keep in mind that wireless networks have very high latency. That means it’s very expensive to create a network connection to a server. If the size of the file you want to download is very small, then you will not gain much with the approach above since it will take almost as long time to get the header information as downloading the entire file.

written by Nick \\ tags: , , , , , , ,

21 Responses to “How To Download a File Only If It Has Been Updated”

  1. iamleeg Says:

    Hi Nick,

    I’ve successfully used If-Modified-Since in an iPhone application. This is what I did:

    NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL: updateURL
    cachePolicy: NSURLRequestUseProtocolCachePolicy
    timeoutInterval: 30.0];
    NSDateFormatter *formatter = [[NSDateFormatter alloc] init];
    NSLocale *usLocale = [[[NSLocale alloc] initWithLocaleIdentifier:@”en_US”] autorelease];
    [formatter setLocale: usLocale];
    [formatter setTimeZone: [NSTimeZone timeZoneWithName: @"GMT"]];
    [formatter setDateFormat: @"EEE, d MMM yyyy HH:mm:ss"];
    NSString *dateString = [[formatter stringFromDate: updatingList.lastModified] stringByAppendingString:@” GMT”];
    [formatter release];
    [request addValue: dateString forHTTPHeaderField: @"If-Modified-Since"];
    urlConnection = [[NSURLConnection alloc] initWithRequest: request
    delegate: self];

    (I’m not sure now why I didn’t format the time zone in the usual way…) anyway, I found in this particular case that I got 304 and 200 status returns as you would expect.

  2. Jorge Says:

    Very useful post. Thanks for putting this together. I’ve implemented this pattern for downloading information kept in a database, but never for files.

  3. Nick Says:

    @iamleeg: Great! Thanks for the code snippet.

  4. Paul Styrnol Says:

    Shouldn’t that be a NSMutableURLRequest followed by a call to setHTTPMethod:?

  5. Nick Says:

    @Paul: If we had to change the HTTP method, then a NSMutableURLRequest would be appropriate. But in this case a default GET request was fine.

  6. Paul Styrnol Says:

    A GET request works in the sense that it allows you to check the response headers but it does download the file. So the code you posted will *always* download the file, ignore the response body and if the file is newer it will do a second download. I don’t understand why you state that GET would be fine if you write about doing a HEAD request yourself.

  7. Nick Says:

    @Paul: You are absolutely correct. I was writing about one piece of code and pasted another. I’ve updated the post with the correct code. Thanks for noticing, and being persistent. :)

  8. bob Says:

    it should be “error:NULL]” instead of “error:nil]”

  9. Nick Says:

    @bob: Technically you are correct. In practice NULL and nil are the same for all intents and purposes. This is one of the better posts explaining the difference. I’ve updated the code accordingly.

  10. Roland Says:

    This looks like it fetches the file each time and checks for the last modified header. Why not include the “if-modified-since” header in the original request? If the file does not need to be downloaded, the server will return a 304 code and will not return the contents of the file.

  11. Nick Says:

    @Roland: The code in the post only downloads the HTTP header because of the “HEAD” request method.
    Before going down the “Last-Modified” path I did some research and found the issues with “If-Modified-Since” linked to in the post above. Based on those issues I did not pursue that “If-Modified-Since” path further. In theory that might have been a slightly better approach because the 304 response is probably smaller than the full HTTP header.

  12. Roland Says:

    Aha! Missed the “HEAD” portion, apologies. Interesting approach, will be trying this. Nice work!

  13. Gary Morris Says:

    Nice! …but don’t forget the [df release] for the NSDateFormatter.

  14. Nick Says:

    @Gary: Thanks for spotting that. Code updated.

  15. Hussein Says:

    nice work

  16. Ram Says:

    Hi ,I am using http post method …. if the update is available i am downloading data..else…i cancel the connection is it correct
    -(void)connection:(NSURLConnection *)connection didReceiveResponse:(NSURLResponse *)response
    {
    NSHTTPURLResponse *httpResponse = (NSHTTPURLResponse*)response;
    if ([response respondsToSelector:@selector(allHeaderFields)]) {
    NSDictionary *dictionary = [httpResponse allHeaderFields];
    // lastmodified= [[response allHeaderFields] objectForKey:@”Last-Modified”];

    BOOL check=[self checkUpdate:[[httpResponse allHeaderFields] objectForKey:@”Last-Modified”]];

    NSLog(@”%@”,[dictionary description]);
    if (!check) {

    [connection cancel];
    [delegate display];
    }

    }
    [webData setLength: 0];
    }

  17. WolWer1nE Says:

    Thanks, helped me a lot!
    Very nice post!

  18. Mustafa Says:

    The HEAD request isn’t properly working for me.

    I get (null) for “Last-Modified” key. Printing out the value of AllHeaderFields, it shows that there’s no such key. Here’s what AllHeaderFields dictionary contain:

    AllHeaderFields: {
    “Cache-Control” = “max-age=0″;
    Connection = “keep-alive”;
    “Content-Encoding” = gzip;
    “Content-Type” = “text/plain; charset=ascii”;
    Date = “Fri, 17 Feb 2012 12:44:59 GMT”;
    Etag = 19202n;
    Pragma = public;
    Server = dbws;
    “x-robots-tag” = “noindex,nofollow”;
    }

    Any pointers?

  19. fausto Says:

    I’m a novice… and I start now to know programming. I’ve seen this tutorial and is near what I need: I have to make a simple button that when press end start the download of a file (.epub) from a server (I define in the code the name and path of file to download) saving it on the iPhone. Can you give me the exact code to do this and where I have to put it? Thankkkkkkkssss

  20. Nino Says:

    Hey, great code!
    Helped me a lot with my project, thank you!!!!

  21. Jeanette Says:

    thanks a lot. This code to get the head information was very useful

Leave a Reply