Files
ppsspp/Common/File/FileUtil.h
T
Henrik RydgårdandClaude Opus 5.5 9ddfcca670 Firmware install: Fewer and parallel file operations into the staging directory
Through Android's storage framework, every exists check, create and open is a round trip to the
storage provider, and writing each file took four of them (a check of its directory, a check of
the file, the create, the open). That made a 6.61 install take around 30 s on a phone, against a
fraction of a second of decrypting and inflating.

The install unpacks into a staging directory it has just cleared, so nothing in it can exist yet.
With the new outputDirIsEmpty option, each directory is created once and each file is created and
opened without checking for it first (File::WriteDataToNewFile), on the IO threads while decoding
goes on. A name that turns up twice still goes the careful way, overwriting the first. The swap
into place at the end is unchanged.

Headless --unpack-updater uses the same path when the output directory doesn't exist yet. Its
output is identical either way. With SIMULATE_SLOW_IO, unpacking the fonts went from 17 s to 7.5 s,
most of what's left being the fixed cost of opening the updater.

The UI's install already runs on a thread pool IO thread, so writes queued on the same pool
could stall behind other long IO tasks, and the pool drops queued tasks unrun at teardown,
which would have left the install waiting forever on its in-flight count. Eight threads owned by
the writer (attached for JNI) avoid both, and headless no longer has to start the pool for it.

Also compare file names case-insensitively when looking for repeats, since the storage usually
is case-insensitive and a second create there makes a renamed copy.

Checked with ThreadSanitizer, and on a read-only destination (every file fails, no hang).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-19 16:06:47 +02:00

269 lines
8.6 KiB
C++

// Copyright (C) 2003 Dolphin Project.
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, version 2.0 or later versions.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License 2.0 for more details.
// A copy of the GPL 2.0 should have been included with the program.
// If not, see http://www.gnu.org/licenses/
// Official SVN repository and contact information can be found at
// http://code.google.com/p/dolphin-emu/
#pragma once
#include <cstdio>
#include <string>
#include <string_view>
#include <time.h>
#include <cstdint>
#include "Common/File/Path.h"
// Some functions here support Android content URIs. These are marked as such.
#ifdef _WIN32
inline struct tm* localtime_r(const time_t *clock, struct tm *result) {
if (localtime_s(result, clock) == 0)
return result;
return NULL;
}
#endif
namespace File {
// Mostly to handle UTF-8 filenames better on Windows.
FILE *OpenCFile(const Path &filename, const char *mode);
// Reminiscent of PSP's FileAccess enum, due to its use in emulating it.
enum OpenFlag {
OPEN_NONE = 0,
OPEN_READ = 1,
OPEN_WRITE = 2,
OPEN_APPEND = 4,
OPEN_CREATE = 8,
OPEN_TRUNCATE = 16,
// EXCL?
};
// TODO: This currently only handles Android Content URIs, but might port the rest
// of DirectoryFileSystem::Open here eventually for symmetry.
int OpenFD(const Path &filename, OpenFlag flags);
// Cross-platform way to close FDs, corresponsing in platform support with OpenFD above.
void CloseFD(int fd);
// Resolves symlinks and similar.
std::string ResolvePath(std::string_view path);
// Returns true if file filename exists
bool Exists(const Path &path);
// Returns true if file filename exists in directory path.
bool ExistsInDir(const Path &path, std::string_view filename);
// Returns true if filename exists, and is a directory
// Supports Android content URIs.
bool IsDirectory(const Path &path);
// Returns struct with modification date of file
bool GetModifTime(const Path &filename, tm &return_time);
// Same but with unix timestamp
bool GetModifTimeT(const Path &filename, time_t *return_time); // the time_t, of course, matches time_now_unix_utc().
// Returns the size of filename (64bit)
uint64_t GetFileSize(const Path &filename);
// Overloaded GetSize, accepts FILE*
uint64_t GetFileSize(FILE *f);
// Computes the recursive size of a directory. Warning: Might be slow!
uint64_t ComputeRecursiveDirectorySize(const Path &path);
// Returns true if successful, or path already exists.
bool CreateDir(const Path &filename, bool quiet = false);
void ChangeMTime(const Path &path, time_t mtime);
// Creates the full path of fullPath returns true on success
bool CreateFullPath(const Path &fullPath);
// Deletes a given file by name, return true on success
// Doesn't support deleting a directory (although it will work on some platforms - ideally shouldn't)
bool Delete(const Path &filename, bool quiet = false);
// Deletes a directory by name, returns true on success
// Directory must be empty.
bool DeleteDir(const Path &filename);
// Deletes the given directory and anything under it. Returns true on success.
bool DeleteDirRecursively(const Path &directory);
// Renames/moves file srcFilename to destFilename, returns true on success
// Will usually only work with in the same partition or other unit of storage,
// so you might have to fall back to copy/delete.
bool Rename(const Path &srcFilename, const Path &destFilename);
// copies file srcFilename to destFilename, returns true on success
bool Copy(const Path &srcFilename, const Path &destFilename);
// Tries to rename srcFilename to destFilename, if that fails,
// it tries to copy and delete the src if succeeded. If that fails too,
// returns false, otherwise returns true.
bool Move(const Path &srcFilename, const Path &destFilename);
// Move file, but only if it can be done quickly (rename or similar).
bool MoveIfFast(const Path &srcFilename, const Path &destFilename);
// creates an empty file filename, returns true on success
bool CreateEmptyFile(const Path &filename);
// Can we actually create files in this directory? Checks by trying, since permission bits
// don't tell the whole story (Windows ACLs, read-only mounts, ...).
bool IsDirectoryWritable(const Path &path);
// Set or clear a file's read-only-ness, which is what a FAT read-only attribute maps onto.
// Returns false where the platform can't express it - notably Android content URIs.
bool SetFileWritable(const Path &filename, bool writable);
// Opens ini file (cheats, texture replacements etc.)
// TODO: Belongs in System or something.
bool OpenFileInEditor(const Path &fileName);
// Uses some heuristics to determine if this is a folder that we would want to
// write to.
bool IsProbablyInDownloadsFolder(const Path &folder);
// TODO: Belongs in System or something.
const Path &GetExeDirectory();
const Path GetCurDirectory();
// Portable version of fseek() that works with 64-bit offsets if supported by
// the operating system.
int Fseek(FILE *file, int64_t offset, int whence);
// Portable version of fseek() that works with 64-bit offsets if supported by
// the operating system, and returns the new offset from the start of the file
// or -1 if it failed.
int64_t Fseektell(FILE *file, int64_t offset, int whence);
// Portable version of ftell() that works with 64-bit offsets if supported by
// the operating system.
int64_t Ftell(FILE *file);
// simple wrapper for cstdlib file functions to
// hopefully will make error checking easier
// and make forgetting an fclose() harder
class IOFile {
public:
IOFile() {}
IOFile(const Path &filename, const char openmode[]);
~IOFile();
// Prevent copies.
IOFile(const IOFile &) = delete;
void operator=(const IOFile &) = delete;
bool Open(const Path &filename, const char openmode[]);
bool Close();
template <typename T>
bool ReadArray(T* data, size_t length)
{
if (!IsOpen() || length != fread(data, sizeof(T), length, m_file))
m_good = false;
return m_good;
}
template <typename T>
bool WriteArray(const T* data, size_t length)
{
if (!IsOpen() || length != fwrite(data, sizeof(T), length, m_file))
m_good = false;
return m_good;
}
bool ReadBytes(void* data, size_t length)
{
return ReadArray(reinterpret_cast<char*>(data), length);
}
bool WriteBytes(const void* data, size_t length)
{
return WriteArray(reinterpret_cast<const char*>(data), length);
}
bool IsOpen() const { return nullptr != m_file; }
// m_good is set to false when a read, write or other function fails
bool IsGood() const { return m_good; }
operator bool() const { return IsGood() && IsOpen(); }
FILE* ReleaseHandle();
FILE* GetHandle() { return m_file; }
void SetHandle(FILE* file);
bool Seek(int64_t off, int origin);
uint64_t Tell();
uint64_t GetSize();
bool Resize(uint64_t size);
bool Flush();
// clear error state
void Clear() {
m_good = true;
#ifndef HAVE_LIBRETRO_VFS
#undef clearerr
std::clearerr(m_file);
#endif
}
private:
FILE *m_file = nullptr;
bool m_good = true;
};
#ifdef HAVE_LIBRETRO_VFS
// Call this on libretro core initialization with the libretro virtual file
// system interface.
void InitLibretroVFS(const struct retro_vfs_interface_info *vfs) noexcept;
#endif
// TODO: Refactor, this was moved from the old file_util.cpp.
// Whole-file reading/writing
bool WriteStringToFile(bool textFile, std::string_view str, const Path &filename);
bool WriteDataToFile(bool textFile, const void* data, size_t size, const Path &filename);
// For a file that's known not to exist yet. On Android content URIs, this saves the existence
// check WriteDataToFile makes, a round trip to the storage provider - but if the file does exist,
// the provider makes a renamed copy next to it instead of overwriting it.
bool WriteDataToNewFile(const void *data, size_t size, const Path &filename);
bool ReadFileToStringOptions(bool textFile, bool allowShort, const Path &path, std::string *str);
// Wrappers that clarify the intentions.
inline bool ReadBinaryFileToString(const Path &path, std::string *str) {
return ReadFileToStringOptions(false, false, path, str);
}
inline bool ReadSysTextFileToString(const Path &path, std::string *str) {
return ReadFileToStringOptions(true, true, path, str);
}
inline bool ReadTextFileToString(const Path &path, std::string *str) {
return ReadFileToStringOptions(true, false, path, str);
}
// Return value must be delete[]-d.
uint8_t *ReadLocalFile(const Path &path, size_t *size);
} // namespace