match object(src) {
        Ok(res) => return Ok(res),
        Err(JSONParseError::NotFound) => {} // if not found, that ok
        Err(e) => return Err(e),
    }

You probably have realized that this is really tedious, and this is where macros would really shine:

    macro_rules! try_parse_as {
        ($f:expr) => (
            match $f(src) {
                Ok(res) => return Ok(res),
                Err(JSONParseError::NotFound) => {} // if not found, that ok
                Err(e) => return Err(e),
            }
        );
    }

    try_parse_as!(object);
    try_parse_as!(array);
    // ...

It is also possible to avoid macros by translating `Result<(&str, JSONValue), JSONParseError>` into `Result<Option<(&str, JSONValue)>, JSONParseError>` (where `Ok(None)` indicates `Err(JSONParseError::NotFound)`), which allows for shorthands like `if let Ok(res) = translate(object())? { ... }`.

Also, even though you chose to represent numbers as f64, a correct parsing algorithm is surprisingly tricky. Fortunately `f64::parse` accepts a strict superset of JSON number grammar, so you can instead count the number of characters making the number up and feed it into the Rust standard library.